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TRAINING RESEARCH PROGRAM AND PLANS: 
ADVANCED SIMULATION IN UNDERGRADUATE PILOT TRAINING (ASUPT) 



I.O INTRODUCTION AND BACKGROUND 

— '■ — th o H kjr n an^ L abo ratory^ Flying lr a \iv U\^Wvl&lor^-4M^^/^ 

has developed a unique capabil ity for c ondu cttng rese arch oh Important 
methods and equipment problems fn pilot training. Although^ the present 
orientation of the laboratory is toward the solution of undergraduate 
pilot training problems many of these study results will be general I zable . 
to other training programs. Further, the. f lexlbl I Ity of the HRL/jFT 
facility makes It possible to carry out Investigations of many problems 
of training methodology and equipment unique to other pilot training 
programs, / ^ 

In planning the HRL/FT facility, contract studies were conducted 
to specify the training problems to be Investigated and the organization 
and equipment requirements for conducting these Investigations, The 
contract results were reviewed and revised, by Air Force personnel arid 
formulated Into a basic working document entitled "ASUPT Utilization 
Concept", Under this concept Investigations were centered primarily up- 
on the training problems and training device characteristics necessary 
for an effective undergraduate pilot training program to be carried out 
by HRL/FT, Consideration was also given, although not so expl Id ty, to 
the training problems peculiar to prograrrp beyond undergraduate pilot 
training. 

As Indicated, Human Resource^ Laboratory research tnanagement planning 
focused first upon research designed to Increase the effectiveness of 
the Undergraduate Pilot Training program (UPT), However, management 
was equally concerned that, to the maximum extent possible, the research 
be carried out In a way which woul^ make the results genera I Izab le to 
other levels of pilot training. These considerations led to the de- 
cision to initiate a contractor conducted study to Identify these 
general training research questions. These questions were Identified 
through conduct of a survey of. those Individuals In the training 
community selected for their experience with the field. This panel of 
experts was composed of those Individuals In government and Industry 
who were recognized as experjenced and knowledgeable and who coul^ de- 
vote the necessary time to the program. The panel members were asked 
to Identify those research questions which they felt to be* pressing In 
terms of their Impact upon pilot training effectiveness, Tl\ey were 
asked, also, to order these quest I.oins terms of their Nportance to 
Increasing the effectiveness of* begfnrvlng pilot training programs. 

The study reported here Is the result of work toward the goal of 
defining those priority research Ihvestlgatlons to be^undertaken 




using f^e research capabilities of HRL/FT^ Specifically the study en- 
compassed the following major elements: 

I. A statement of the major training research questions. 

2 An-a ssl gnment-of -p r? or! t } es - by knowledgeable Individuals 
_ to. these research questions In terms of their Importance 
±Q_}.n crflf^5?l«ng tr a lnln ij ef fectU^eness ^ ^ 

3. Examination of the research questions In light of the 
research capabilities of HRL/FT, 

4.. Restructuring the priorities based upon the research 
capabilities and constraints. 

5. Setting forth the details of those experiments and 
Investigations to be undertaken most Immediately. 



2. a METHOD 



The method used ^n carrying out the work may be described under 
three major categories of effort. These were (a) development of a 
priority list of research questions In terms of their Impact upon train- 
ing effectiveness, (b) examination of the capabilities of the research 
facility and, (c) delFneatlon of the high priority experiments, and In- 
vestigations to be carried out using the faclllfy. 

2.1 Development of a Priority List of Re search Questions 

Tjie development of the list of research questions and the deter- 
} . ml nation of the priorities of the Items on the list was carried out In 
^ ' two phases. ' During the first phase a list of research questions was 
generated through review of the relevant current literature, and most 
importantly, by a panel of selected Individuals. These Individuals were 
generally recognized as experts In the fJeld of pilot training and 
research. Their background' ref tected exper'Ience In governmer>t agencies, 
unlversrtles ind private Industry^. These panel members were contacted 
Jn person or by ^telephone ^nji a* foTlow-up letter explaining the project 
In detail, was sent. A copy of this letter Is given Jn Appendix A. 

Twelve panel members submitted researolf c^uestlons which they believed 
to be most relevant to be Investigated In order to Increase the effective- 
ness of pilot training. ' In all; a list of fifty-five research questions 
was generated. These research questlon-s were not equal with respect to 
the .breadth or scope of effort Which would be required. They were, how- 
ever, those questions which were deemed important enough by this group of 
experienced Individuals to be. Included In q research program. These 
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questions were defined and explained as concisely and clearly as (>osslb e 
and re-submltted as a total list to each of the indlvfduats on the panel 
for their Judgments as to the priority rankings of the questions. The 
definitions of the research items are given In Appendix^B. — =Ftie nstruc- 
tions to the raters tor assignjjig priorities are given In Appendix C. 



The method suggested to the paneTTmembers tor assTgrttng priori I los 
|-compar+son-tech^iq<je. This technique requlmi 



that each item be listed on a 3x5 card. The rater sejected one item and 
then sorted the remaining items as to whether each item was of greater 
priority or lesser priority than the item selected. This procedure re- 
sulted in two groups of items; one of higher priority and another of lower 
priority than the selected Item. Each of these groups were then so;-ted 
in a similar manner and the resulting groups further sorted in the same 
way until groupl of four or five items were attained. These items were 
then ranked as a group and, finally, ali items assembled into a priority 
'list. Ten panel members assigned priorities, two being unable to because 
of other commitments. The results of these listings and the priority 
ratings are given and discussed In Section 3.0. 

2.2 Examination of the Capabilities of the Research Facility 

The objective of this phase of the contract effort was to examine 
the AFHRL/FT research capabilities and constraints so that particular - 
research Investigations could be detailed and recommended. These recom- 
mendations could then be based on these^capabl I Itles and constraints, as 
well as the results from the panel of experts. 

The Items of research equipment at the AFHRL/FT facIIItyVll be 
Identified here only In general terms since they are described In detail 
In. Hagin and Smith (1974). 

The major Item of research equipment Is the- Advanced Simulation In 
Undergraduate Pilot Training (ASUPT) device. ASUPT has'come to mean the 
large scale two cockpit full motion capability simulator with a 7 channel 
computer generated visual sysfem. This device Is described on pages 17 
through 26 In HagIn and Smith To summarize. It has a six degree of 
freedom motion system and an .external world -visual system with a field 
of view which essentially dupillcates that of the T-37 aircraft which It 
represents. The Computer Image Generator (CIG) produces a very extensive 
variety of objects for making up the visual scene for the pilot. 

The ASUPT device incorporates a number of advanced instructional 
features which may be tested for their contribution to teaching effective 
ness. These capabilities include selective task sequencing; variable 
ta^k difficulty and complexity; selective malfunction insertion; , 
simulator freeze; rapid .re-initial ization; automated demonstration; self- 
confrontation; and a number of methods for providing knowledge of results 
to the trainee. 
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^ The total HRL/FT training research facility Includes several de- ' 
vices less complex t/han the ASUPT which makes It possible to Investl'gate 
certain research questions using these' less con^pllcated and expensive 
devices. The first of these Is the T-4G trainer. This de>^;ce Is a two 
degree of freedom motion platform tralnerj with a Slng6r SPDxJectronlc 

- Persf)ecMve Transformati on (EPT)^ visu a l system. The Image gerfer^tlOTi 

system, based on a color movie film, uses a CRT to project an electr^on- 

t ca^liy -generated 4^orf zon— d4s^lBy^iichH4>at— on— dpproach7-fand+^ ^n^^ke-^ 

off sequence may' be presented as 'a. visuaj diVplay for the trainee. 

Another Important Item of research equipment Is the T-40 trainer. 
This fralner Is an Instrument and procedures trainer Incorporating a two 
degree of freedom motion system. 1t Is a side-by-slde seating co<ikpIt 
with an instrument configuration similar to that Installed 'in the T-39 
aircraft.^ 

A further research capability in the H^L/FT facMIty Is the Formation 
Flight Trainer (FFT). This trainer Is a fixed base lo^ fidbllty part- 
task trainer which provides some of the essential visual cum for teaching 
undergraduate pilots the basic formation flight skills. The^cockplt has 
a stick, throttle, rudder pedals and-slmple Instrument displays and the 
trainer Incorporates the rudimentary f light dynamics of the T-38 aircraft. 
A spherical screen Is used to provide a 200 degree horizontal and 90* de- 
gree vertical field of view upon which Is projected an Image of either a 
T-37 or T-38 model aircraft. The trainee Is given the opportunity to 
practice flying formation from observation of the projected Image of this 
aircraft on the screen. ^ . 

An Important capability of the HRL/FT research facility Is the Auto* 
matic Data Acquisition and Control System (ADACS), used In connection with 
the T-4G, T-40 afflf FFT trainers. This Is* a system which has as Its basic 
unit a System Engineering Laboratory (SEL) 72 computer. The ADACS Is \ 
capable of sampling data from the J-40 trainer, the T-4G and the FFT and 
processing It for up to 29 measurenr>ent parameters. The ADACJS permits 
accomplishment of research In the area of recording techniques, perform- 
ance measures and proficiency measurement using both on-line and off-line 
techniques.* The ADACS facility can be Interfaced with the -SEL 86 com- 
puter^ which are the cbre computers fcr the ASUPT device. Together the 
SEL 8d computers and the SEL 72 comprise a powerful computational cap- 
abllltXfor the HRL/FT facility; ' • 

DtheiL^apabl I Itles within the HRL/FT facility support research both 
In the .deter]^ I nation of training device requirements and In training 
methods^ and training device utlJJzatlon. The flrst^of these Is an.i»/e 
movement recording capability wh^'lch has been developed by HRL/FT pSnonhei. 
The device can used both \n aircraft and simulators and Is designed 
to provide data wtth-4^spee+— to-^e-4n^rmat-km-or 
of the display used byVthe-pI lot and the required field of. view. ^ 

' s 
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second capability is the laboratory's audio-visual instructional media 
facilities. These are comprised of a video laboratory for the pro* 
ductlon of video Instructional presentations, a series of learning center 
carrels and the Audio-Visual Instrument Training device (AVIT) developed 
b y Life Scie nces. Inc. This latter device has the capability for pre- 
senting both visual and audio Information, forward and backward branch- 
ing in both audio and visual modes and interaction with tjpe device by the 
student both with af-rcr^ft controls and multiple choice answer response 
keys. ' 

A final major and important piece of equipment available to the 
HRL/FT facility is an Instrumented T-37 aircraft. This aircraft is a 
very Important piece of equipment for obtaining basic information 
relevant to the pilot's overall -SYS*^'" performance and ^s essential to 
the test -of the performance equivalence thesis discussed In Section 4.O. 
.Most importantly, it allows for the correratTorTor^lfe— t^-^eT-O' ^ 
pilot's performance In controlling system output parameters such as 
heading and airspeed with control input parameters-such as fore-and-aft 
and lateral stick movement^. These measures are essential to establish- 
ing a data base for the determination of the equivalence of the ASUPT 
device to the aircraft being sirjfulat&d. ^ 

2. 3 DeMneation of the High Priority Experiments 

» >- 

Ba§ed upon the priority assignrftQnt by the panel of experienced 
Individuals and the capabilities of the research equipment and constraints 
surroundfng the research facility, certain experiments and supporting 
Investigations have been delineated. In the delineation of these ex- 
periments. It was felt'thaf the highest effectiveness possible should be 
made In the use of the. research equipment and" personne I . Therefore, a 
second grdup of individuals with experience in experimental laboratory 
and field investigations was selected to provide expertise in how these 
experiments should "be conducted. It was a function of this panel to 
provide guidance in designing these .experiments so that they would be 
most economical and most effective in attaining results which could be 
used with confidence in decisions with respect to training procedures 
and training methods.' These experiments are delineated in a later section. 

It was also felt necessary to examine the research questions, 
particularly those having to do with training methodologies .and progression 
of the trainee through the training program, utilizing le^s complex and 
expensive devices to the maximum extent possible. It was felt that much 
information directly useful for improving the effectiveness of the Under- 
graduate Pilot Training program could be gained through direct use, and 
experimentation with the simpler devices. For example, eafrlier work with 
the use of the T-4G simulator in the T-37 Undergraduate Pi lot Training 
program has demonstrated the worth of such devices in determining the 
contribution of different training approaches, (see Woodruff, Smith i 
Morris, 1972). They have also proved valuable in the investigation of 
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system performance measures and are recommended to be put to further use 
In the Investigation of operator output performance measurement. The 
audio-visual capability of HRL/FT along with devices such as the AVIT 
are Important to Investigations of training In specif Ic -training objec- 
tives which form the basis for accomplishing the final training objective 
for the undergraduate pi lot, 

3.0 PRIORITY RAtlNGS OF RESEARCH PROBLEMS 

The priorities assigned by the panel of experienced training research 
Individuals were used to generate an Initial listing of problems Judged 
to be important to Increasing the effectiveness of pi lof tralalng. This 
list, taken In conjunction with the capabilities of the HRL/FT facility, 
provided the basis for derlving^a list of recommended Investigations. 

As"^as described earlier, the members of the panel were asked to 
arrive at a listing of the Items uslhg-a modified paired comparison 
technique such that high priority Items could be Identified. This 
technique Is described In detail In the Instructions to the r^rs given 
In Appendix C, ; 

^ & , * 

The median ratings for tfte^ Items were computed and are tabled In 
Appendix D, along with their sejpl-lnterquarti le range (Q) values. An 
abbreviated list In Judged order of Importance Is given In Table 3,1, 
This rist Is derived from the panel list and consi'deratlons of adminis- 
trative, scheduling and equipment constraints. It will be discussed In 
detail In later sections. 

Tab le 3. 1 

^ Abbreviated List of F| rst Priority Research Questions 

Formula'ted on Basis of Multiple Criteria 

1. Motlon-Vlslpn Interaction, * 

2. Content of the Visual Display 

3. Sequencing of Training Tasks 

4. - Cognitive Pre-Tralning . , . 
^ 5. Individualized Instruction ' ^ 

6. Feedback 

7. Instructor Training 

8. Contextual Training 



In considering, the Items In these lists an examination of the semi-- 
interquartile range values listed under the Q column Is In order. These 
values reflect the variability among the ratings for the several Items. 
They show considerable variability In the ratings of the Items given by 
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the panel members. To a great extent, this variability arises from 
genuine differences of opinion among the raters as to which problems are 
more Important for solution in improving, the effectiveness of pilot 
training. However, some of the variability comes from the seemingly in- 
escapable Intrusion of other criterion into the rating procedure. Since 
the panel members have been working first hand with* on-going training 
programs and carrying out training research in operational settings, 
•they found It difficult in some cases to assign priorities on a purely 
detached "contribution to training effectiveness" basis. For example, 
some JOdges felt that certain items required more effort in^pulllng to- 
gether relevant information In the literature and in applying that 
information to the pro.blem. The feeling was that if this were done, 
the Item would not command a high priority in an on-going research 
program. Some Judges felt that certain items were so important on other 
bases, such as pilot acceptance, that it would not be worthwhile to ex- 
pand effort on evaluating them in^terms of contributions to training £er 
se. Again, souie^Judges felt that for certain Items such as aircraft 
dynamics simulation, the engineering developments were such that research 
as to its relationship to training effectiveness could not be considered 
high priority. That is to say, that providing high fidelity in the air- 
craft dynamics simulation was felt to be so feasible and relatively 
economical that research on the fidelity of aircraft dynamics simulation 
was not particularly urgent.* 

The comments and annotations by the panel of Judges In assigning 
the priorities to the research items are important in interpreting the 
priority list. However, those items which appear in the top ten of the 
list in Appendix D are not dissonant with the general conclusions to be 
drawn from discussions with the individual Judges. This is «lso true of 
the low priority Items in the list. 

Ift was indicated in Section 1.0 that the research investigations to 
be undertaken as first priority, would be selected on the basis of the 
research priorities assigned by the training problem and other consid- 
erations. These additional considerations are: the capat)i I ities of the 
research facility, th« critical ity of a question in terms of timely In- 
put into the training change process, and administrative considerations 
such as subject and reseaj^-personnel aval labi I Ity and practicality in 
scheduling, .these con si delations, taken along with the experts' ratings, 
form the basis for an overall plan for undertaking investigations and 
the recommending of certain first priority experiments. The plarf and 
the experiments are described in Section 5.0. , • 
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- 4.0 METHODOLOGICAL CONSIDERATIONS ' 
4.1 General Considerations. / 

There are certain considerations which It \s felt should be dis- 
cussed before arriving finally at a plan for long range training' 
research. The first of these are the di fficu Ities ' inherent in carrying 
out classical transfer of training experimerits. These difficulties make 
It advisable to examine alternate .methbds for obtaining experimental 
data for use in making decisions about training methods and training 
.devices. The development and test of methods which may provide useful 
data more economically and in niore timely fashion is an area which is 
recommended as worthy of investigation because of the potential savinqs 
In money and time. , ^ 

The difficulties attendant on classical transfer studies are time 
cost and availability of experimental subject trainees. In order to ' 
assess adequately the transfer effects of any given variable under study, 
a representative sample of a class of student^ is required for each of 
the experimental groups and the control group. The calendar time re- • 
quired for an experiment can vary from a short period when the effect of 
Training In the simulator upon the performance of a' single task is 
assessed to that time requir6d to train a class in a complete curriculum 
as would be the case when investigating different sequences of the train- 
ing tasks. When the experimerjtal question is that of the configuration 
of the training equipment the methodological problems are somewhat diff- 
erent from those in which the research addresses training methods- and 
equ pment at1 lization. Let us first examine the training methods and 
equipment utilization research area, since it is here that the classical 
transfer of training methods are most applicable and will need to be 
preserved. 

The primary problem in the classica I transfer approAh lies In the 
cost and caie.ndar time required for transfer to the aircrSft after 
training in the simulator. In order to expedite the acqujsltion of 
Information relevant to the effectiveness of different frain'fng methods 
and ways of utilizing the simulator, it is proposed that studies be 

t,^b 1 1 sh the full ASUPT simulator as a criterion system 
to be used In those transfer of training studies Investigating training 
methodology and simulator utljlzatlon. ^ » y 

4. approach, J t is essential that the ASUPT device be ^ 

established as a vallj representation of the aircraft. In the past this 
nas been done. In two major complementary ways. The first of these Is 
to Implement the mathematical model of the flight characteristics of the 
aircraft as.accurately os' possible in the simulator computer; "The 
!!T i !! general cue environment of the pilot as much like 

that of the aircraft as possible using the cr'iterion of asking experi- 
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enced pilots to fl/both the simulator and the aircraft and to make 
judgments as to thClr similarity. It Is proposed that this technique 
may be strengtherJd by the use of measurements of the pilots' control 
perfonwnce wh4 Us flying both the aircraft and the simutator. It Is 
proposed that a/ estimate of the population distribution of these per- 
formance value/ for the aircraft be established through collection of 
data In an ln#rumented T-37 aircraft. The rationale Is that, If the 
experienced fwlot tiles the aircraft through maneuvers In which specified 
system measu/es are required to be within specified limits and It Is 
shown that ms control performance In the ASUPT while performing within 
Identical llmlfs Is the same as In the aircraft, thls^ls supportive data 
to the equivalence of the simulator to the aircraft 
to pilot jpntrol. The outline of a data collection 
craft Is'yjolscussed In Section 5.2.5. 




Important 
the air- 



The method being proposed has been termed the performance equiva- 
lence a&proach to estab llshlng +he characteristics of the training 
simulator. The method constiltutes a^ quant I f Icat Ion of the present 
metho^ for jjsing the ski II s\ and. knowledges of highly trained and 
expei/enced pilots In maklngWhe simulator equivalent to the aircraft 
in tjSrms of the pilot perfomfance required. Establishing The utility 
and /Validity of this technlqub In the ASUPT Investigations will provide 
the/ground work for Its use In the* eva I uat I on and calibration of other 
trafinlng simulators- within thd Air Force. 



at 



Performance equivalence I 
be established upon demonst 
conditions Is not different 
fet of conditions. That Is to 
Jutput obtained in the Instrume 
of the population mean Cx) and 
//for specified measures. Confide 
/mean to determine Whether any ot 
figurations of the simulator, fa I 
limits with respeqt to the perfo 
necessary to point out that the 
be established as being reliable, 
meaningfully related to the task 



terms of operator performance Is assumed 
Ion that his performance under one set 
rom his performance under a reference 
ay that performance measures of operator 
ited aircraft may be used as an estimate 
le s'tandard error about that mean (^x) 
I i Imtts may then be set up about the 
r set of conditions. I.e., con- 
w 1th in or without those confidence 
mce measures, bei-ng taken. It Is 
formance measures In question must 
scrlmlnating, and logically and 
ihand. 
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Once the ASUPT system has been 4stabl Ished as being representative 
of the criterion aircraft and performance equivalence has been estab- 
lished, the system may be utilized as the criterion system for transfer 
of training experiments In which the aircraft would normally be used.. 
Also, it provides a criterion system foV research In situations which 
are not practical or advisable in the alVcraft. These are those which 
either pose safety hazards or Impose unusijal stresses on the aircraft. 

The questio!j.of the general Izabl l Ity tof. the findings regarding 
training ftiethods obtained In this manner mdv be raised at this point. 
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■ rf could be held that results as to the comparative effectiveness of 
different training methods obtained through using A'SUPT as the criterion 
system are not valid for training for transfer to the aircraft. If this 
thesis were true it would follow that the results could be held not to 
be valid for use in other simulators as well or that methods used in 
one training aircraft are not useful in another. Either contention 
would seem to be contrary to the. body of evidence and practice in 
education and training In which traiftlng methods and techniques are 
valid across a variety of training situ^ions and training materials 
and are not unique to them. 

The second major area in which training effectiveness decisions 
must be made is that of the configuration of the training simulator. 
Here again, we may consider the use of the classical transfer of train- 
ing paradigm and the role that might be played by other methodologies 
In support of. training effectiveness decisions. When carrying out a 
classical transfer of training study in which the question Is being asked 
as to the relative importance to transfer of training of gi^yen levels of 
the characteristics it is necessary to establish levels of that variable 
for -experimental study; For example, a transfer of training study might 
be carried out in which all combinations of degrees o.f f-reedom of move- 
, ment of the simulator are selected as experimental conditions. The 
numbep-^ possible experimental conditions and tr-aiiiees, as we\ I as the 
amount of calendar time required for the experiment would be v4>ry large 
(see Smith, 1972). Further, and importantly, it is necessary tocarry 
out prior studies to determine whether each of the levels of the\ experi- 
ment as defined in physically measurable terms is meaningful. These 
antecedent studies wo.uld have 'the -goal of determining which of these 
physically defin^ed conditipns are different in terms of the pilots' 
performance. . i * 

It is evident that many characterJL3t ics o f a simulator Ln .which we j 
may have a valid Interest as to fhei r. training effectiveness may be 
defined In physical units which can be differentiated through use ^f the 
appropriate metri.c but which cannot be differentiated by the operating 
pilot at the level of his perception and/or performance. This Ls the 
classical psychophysical problem In which intervals on the psychological 
scale bear-.a relationship to the intervals of the physical scaU, but 
not necessarily one which, is linear or even monotonlc. In training 
effectiveness research the levels of a variable being investigated must 
be set based upon Interval^ „dl f ferentiated in terms of performance rather 
than physical units. The determination of the performance intervals is 
prerequisite to transfer of training research for many variables of 
interest. To Illustrate further the importance of the psychophysical 
relationship we may consider the fidelity of motlpn variable in trainipg 
simulator research. The degree of_fldelity of motion may be defined in 
physically measurable terms in a variety of ways which could be varied 
systematically to-determine its effect upon transfer of training. 
Parenthetically, in choosing the physical ex^ession of a variable one 
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should be selected which is meaningful and useful for the simulaxof de- 
sign engineer. The fidelity of motion variable may be expresseo in terms 
OT the frequency response of the motion ''p I at form to some driving function 
which might be varied over a range of physical values for exper/imental 
purposes. However, in this case, as is true in many others, t^iere is^ 
virtually- no background information to guide the experimenter In setting 
levels of the experimental variable. As stated earlier, it is necessary, 
through essentially a psychophysical procedure, to esta.blish discriminab le 
intervals of the variable before considering further experinventation. 
What must be established are the physical intervals of the variable which 
are different for the operator in terms of his performance. Like the 
fidelity of motion variable, a great many variables of research Interest 
require establishing the psychophysical scale before experimentation 
can proceed. 

What Is proposed Is that prior to the conduct of transfer of train- 
ing experiments, it is necessary to establish, for many experimental 
variables, that the conditions or 'levels of any variable being examined 
are different In terms of operator performance. Conditions or levels 
set up on the basis of the physical measurement of the variable may or 
may not be different in any way meaningful or even perceptible to the ^ 
pilot. Those levels of the experimental variable which do not differ 
with respect .to^ perfortnanee could not be expected to differentiate with 
respect to transfer of training. It Is to be emphasized that establish- 
ing the limits of the .physical scale within which performance is equi- 
valent establishes an interval on the performance scale and that levels 
of , the experimental variables' must be set on the basis of this scale. 

Once the ASUPT has been established as the crl-h&rlon system a less 
costly met{iod than classical transfer can be used for obtaining Infor- 
mation about the effectiveness of simplified systenr/s. The method rests 
on the assumptions that transfer of training from the criterion ASUPT is 
positive and high and that a modification of ASUPT, even though different 
from the crlterloo ASUPT In Its physical metric, when equivalent to it In 
terms of pilot performance will provide equally positive and hlqh 
transfer. Given these assumptions many simulator characteristics which 
are Important to investigate from a cost-effectiveness stand-point may 
be studied without conducting formal transfer' o/ training experiments. 

The minimal system which' is equivalent t</ the criterion ASUPT with 
respect to any simulator characteristic Is just that - the minimal 
essential physical system which Is equlvaleryt In terms of performance 
to the more complex one. It Is assumed that transfer to the aircraft 
would be as great using that system as wftw the criterion system. How- 
ever, other configurations which coul d .be.'quite unlike the criterion 
ASUPT in some of its characteristics migJlt be better training configu- 
rations. That Is, the trainer may be modified In some of Its character- 
istics or additions made to It to enhance Its efficiency as a training 
device. 
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The point just made leads to making the distinction between effi- 
cient training and effective training. Efficient training Is that which 
brings the student to the Immediate training goal mos,t eponomically In 
■the least training time. If the skill attained upon reaching the Imme- 
diate training goal can be applied In a larger training objactlve the 
Immediate goal training Is effective In terms of the larger ^oal. For 
example. If the cues used in approach and landing can be tayght more 
quickly using a highly abstract display we have an efficient Training 
"technique. It wljj. be an effective, technique only to the exten* that the 
skills acquired by that technique can be applied In the larger mc^tre 
realistic situation, I.e., positive transfer takes pldce. 

We may assume that any training which transfers positively to iNrte 
ASUPT criterion system is effective training for the aircraft. Or we\may 
not accept as effective any training method, procedure or equipment unjtlt 
It has been demonstrated that positive transfer to the aircraft Is the. 
case. The choice Is simple but significant with respect to cost. The 
adoption of the ASUPT as the criterion system for Investigation of the 
most efficient training methods and simulator utilization Is believed to 
involve small risk. In determining the minimally essential character- 
istics of the training simulator', the use of performance equivalence 
methods to establish the least common denominator which is equivalent to 
the ASUPT criterion system appears logical ty sound. The experimental 
data to support rt are sparse although experience with simulator con- 
figurations over the years lends It credence. It only needs to be re- 
called that all manner of simulator configurations have been used with 
positive results. What Is needed Is the systemtzatlon of the method 
for varying those configurations to determine which we might settle upon 
for use. The transfer of the training from that configuration seems 
assured. Studies designed to develop and test such an approach are 
proposed In later sections of this report. 

4.2 Performance Measurement 

in assessing the rate of acquisition of skill or the final level of 
skill of an operator of a complex system such as an aircraft, performance 
may be evaluated at two points In the system. These are at the system 
output point through such measures'as heading, airspeed and aj^ftude and 
at the control Input point through measurement of control movements. 
These measurement points (MP) are shown In Figure 4.1. 

It seems a reasonable assumption that an efficient training system 
would establish In the trainee quickly and economically the control 
Input behaviors of the experienced and skilled operator appropriate to 
bringing about the specified system output. TralnJng may be thought of 
as shaping this control output behavior. Knowledge of the level of 
this behavior as training progresses would be useful to the Instructor 
In guiding the student. ^• 
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Figure 4 J Performance -Mejasuretniant "Points (MP) In the 
Man /Machine System' 



When all training Is given In the aircraft,^ i.e., the finral criterion 
system>. concern with tKe control Input as an Index of skill level as 
opposed to system output Is a less Important matter. System outputs are 
available, have been used as measures over a jong period of time and a , 
good d'eaj Is known about them. (See Demaree, et aL, 1964) Concern wfth 
the operator output parameters begins when the desire Is to train In a 
simulator the skills required for precise control of a dynamic system 
such as an aircraft.. It Is then that system output measures may be mis- 
leading since the human operator can adapt his control behavior to bring 
about the same system output over a broad range of different systems^ 
In this case, the simulator may shape control Input behavior not directly 
transferable to control of the aircraft If the simulator differs fn the 
criterion system In significant ways. It may be possible for the operator 
to control the system such that the correct system output Is obtained whit 
at the same time requiring quite different control Input? - inputs which 
would then be inappropriate to the criterion system with resulting riecjuced 
or even negative transfer of training. . • ^ 

The emphasis here on measurement of o'perator output behavior at the 
control input point Is Its use In the performance equivalence approach to 
the Investigation of the necessary characteristics of 'training simulators. 
With this approach the experienced pilot Is a "calibrated" control element 
being observed to determine whether his control output behavior changes, 
with prescribed changes In the system which he is controlling wh^n 
system performance Is malntalneji at a prescribed level. What Is sought 
Is a measure which Is a reliable description of the pllot^s control 



13 



21 



behavior and is sensitive to changets in task conditions such as the 
dynamics of the system ♦being operated or changes fn the 'sources of, his , 
information for cbntrol ling the system. An example of the latter is a / 
change In the dimensions of movement of control of the motion platform 
in an aircraft simulator^ . ' 

I, - \ 

The*behavior' of the human controller in~a complex closecj-;loop systenf 
Is exhibited in a^complex time varying output , through the controls'of the^ 
system. A simple, dlc^ct, and preferably "on-line" summarization of tbis 
outptJt is required. Previous work has shown that such rneasures as the 
power density spectrum, the breakpoint frequency of the* tran?sfo"rm of that 
spectrum and the percentage of power in selected portions' of the spectrum 
are summary measures which reflect meaningful changes in the operator's 
behavior with changes In the conditions of the task and may be useful 
measures. (See Norman/ 197^ and Matheny et al., i974) The relative re- 
liability of such summary measures and their Inter-relatlonshlps need to 
be established^ What is required is an examination and anajysls pf 
pilot control output behavior in the complex control situation across a 
range of conditions of external disturbances to the system and of re- 
quirements for control, i.e., maneuvers or tasks. Systems which have the 
capability for experimental manipulation of variables for, studying this 
behavior and provi di ng answers to practical questfons regarding Simulation 
requirements are ASUPT, T-40 and T-4G simulators, and the instrumented 
T-37 aircraft. The use of these devices Jn developing such performance 
measures and in the overall research program Is discussed In, Section 5.0. 



5.0 RESEARCH PROGRAM 
5.1 Overal I Plan I 

The major elements in an overall plan for a research program In under- 
graduate pilot training are shown in Figure 5.1. This plan includes as a 
first phase development of procedures, methods and measurement techniques 
for use of the research equipment. The second phase includes carrying out 
screening experiments and more formal investigations. The first stage 
serves to provide a technological base for experimentation and Is funda-^ 
mental to research J n the program. It is designed also to produce data 
applicable to other training research programs and to training research 
device procurements. 

The research phase of the program Is divided Into Investigations of 
training methodology and of training simulator requirements. Each of 
these areas Is further divided Into a screening and a validating phase. 
The screening phase is designed to narrow down fhef multitude of variables 
in a systematic way, particularly with respect to the simulate* re- 
quirements research area prior to more Intensive formal experiments. 

It should be mentioned that training metho'Qology and simulator 
requirements are not mutually exclusive areas of research. Certain 
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training methodologies will require the use of particular simulator 
.charai^teri sties. Conversely, obtain simulator requirements will require 
the utilization of certain metbpdologies. It should be noted also that 
the results of screening expernmoBts may form the basis for recommenda- 
tions for improving training eflfedt-i veness without going through tbe 
more formal experimental stage. 1 \ 

\ \ 
The research plan provides ^lor the investigation and test of certain 
less well used investigative methods with the goal of determining the 
utility of more economical experimental approaches and designs. 

The various elements in the overall research plan are discussed in 
the sections to follow. 

r 

' *5..2 ^ Establishing a Technological Base 

Establishing a technological base means simply, that the development 
and test of procedures, measurement techniques, equipment checks and 
the like are necessary antecedents to carrying out a research program. 
The technological base discussed here extends beyond this since it 
recommends a somewhat different approach by proposing to establish the 

. ASUPT as a criterion device for the conduct of investigations and 
experiments. This phase of the program \^ intended to provide the data 
for "the characteristics of a training simulator. This performance 

.equivalence approach was discussed in Section 4.0. 

It should be pointed out also* with respect to .estab I ishing the 
technTologicaJ base that the working out of procedures,' measurement 
techniques and the like are, proposed to be accomplished using specifi- 
cally designated experimental variables as the vehicle for developing 
the procedures and techniques. The working through of such an experi- 
ment is intended to provide practical information for the technological 
base as well as to provide useful data on the variables involved, 
however, limited or preliminary. This experiment is proposed to be one 
In the simulator requi rements' research area. 
'It',, * ' , ' * 

5.2.1 Equipment Familiarization 

The information to be gained and prpcedures to be established under 
this category are cor^cerned 'principal ly with experimental equipment 
characteristics. Here the research personnel will gain familiarity 
with the' problems and procedures In configuring the equipment. 
Generally, the order of experimental equipment configuration Is deter- 
mined by the experimental paradigm adopted^ by the researcher. However, 
in those experiments dealing with simulator requirements using the . 
performance equivalence paradigm the order of experimental conditions 
may be set up to be most efficient in teqjns of equipment change rather 
than overall experimental paradigm. Thlaf^^holds since experienced 
pilots will be used as subjects and will be trained to specified levels 
of system output under given experimental conditions. Under this 

^ 16 , ' ' 
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experlmentai method it Is not critical that the conditions be ordered 
In any particular manner with respect to subject learning. It Is 
possible then to order the experimental conditions In terms of ease of 
transitioning from one equipment configuration to another rather than 
In accordance with some overall experimental paradigm. Information 
obtained as to the most economical way of changing from one equipment 
condition to another^can be obtained during the early equipment familiar- 
ization phase and will prove a valuable time saver later. 

These preliminary studies and Investigations' will afford the oppor- 
tunity for obtaining data on the consistency and reliability of the 
equipment under continued experimental use. Idiosyncrasies within the 
system can be identified so that they may be taken* into account in plannln 
future experiments, 

A necessary concern Is with establishing calibration procedures for 
the Important and necessary parameters within the system during experi- 
mentation? The type and frequency of calibration necessary to keep the 
equipment functioning at a standard level for" experimenta I purposes must 
be establl'Shed during this phase and provides data for the technologJcal 
base. 

An Important function to be accomplished during this phase is the ^ 
determination of how the experimenters and instructors will function at 
the ASUPT console or Instructors' stations in carrying out the experl- \ 
ments. Since the two Instructor stations differ In terms of the infor- \ 
matlon available to the experimenter or instructor, the procedures and ^ 
practices for providing Instructions to the subjects need to be worked ' 
out and function smoothly, 

5,2,2 System Measures ^ 

These measures are those referred to as M2 in Figure 4,2 and reflect 
the performance of the total man-machine system. In aircraft they are 
such parameters as a i rsp^eed, heading, altitude, arid pitch or roll angle. 
It Is necessary that these be developed and specified for each of the' 
research devices such as ^SUPT, the instrumented airplane and the T-40 
and T-4G trainers. The specification of these measures Is important 
throughout all. phases of experimentation although their use differs 
between the training methodology and the simulator requirements areas, - 

In the areas .of precision of aircraft control useful and reliable 
measures have been developed for most maneuvers both In the simulators 
and In the aircraft, CMterla against which performance may be measured 
can also be rather readily established for carrying out the required task 
procedures. 

System output measures are b'elng established by the Human Resources 
Laboratory, Flying Training Division, through contract and "HRL/FT 
personnel efforts. The HRL/FT effort has been summarized by Waag (1974) 
In which he points *ut that the major emphasis In the present performance 
measurement development Is toward developing those measures which reflect 
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performance in the simulator, \.e.^ ASUPT^ Emphasis Is placed upon these 
performance measures since it is assumed that performance In the simulator 
will be positively related to performance In the aircnjft and it is, 
therefore, possible to use the simulator as a criterion device for In- 
vestigating hardware configurations and training strategies, ' The HRL/FX 
in-house work Is also directed toward defining the objectives for any 
part leu Jar maneuver or sequence of training tasks and to develop measures 
for those particular criteria. A major effort is being directed toward 
developing measures which will determihe^the degree to which the critfhjon 
objectives are met;' reflect only the salient characteristics of perfor- 
mance; are useful by the student and the instructor pilot; and are 
generated on a real-time basis so that feedback may be immediate. 

The systems measurement work being carried out by HRL/FT personnel 
provides a sound basis for sei.ectlon of system measures for any given 
experimental investigation. Life Sciences^ efforts have, therefore, 
been directed toward the development of operator performance measures 
rather than system measures. It should be noted here, however, that 
HRL/FT efforts have nqt been limited solejy to systems measures but have 
also been concerned with operator oiltput measurement, t 

5.2,3 Operator Performance Measures - T-40 Experiment 

i 

'Operator performance measurement was discussed In Section 4,2 In 
connection with the performance equivalence experimental paradigm. These 
measures are taken at the controls operated by the pilot ^nd refject his 
inputs into the system. They reflect his direct response' to the stimuli 
presented to him by the system^ They are distinguished from system 
performance records whi.ch reflect his response plus the varlabi I Ity,^ In 
the system through which he is operating. 

Investigations In the T-40 training simulator^ can be deslgned^to 
examine different procedures for summarizing the time varying operator 
output Into the controls, to examine inter-lndi vl dua I differences In 
'these surtimary measures and to determine the stability or reliability of 
these measures overtime. This device coupled wItK the ADACS data re- 
cording system constitutes a most suitable device for preliminary In- 
vestigations of these questions. 

As a general condition of the experiments in wh|.ch operator output 
measures are being examined, the pilot must control the system to a 
prescribed level of system performance at which time his control output - 
behavior IsYecordea, It Is necessary to standardize system performance 
in order that changes Introduced into -the system being controlled may be 
reflected In the operator*s control output. Control output behavior Is 
then compared across changes introduced Into the system or the tpsk 
being carried ouf\ When such changes do not result In changes in the 
experienced operator's control behaylor, the conditions belngsTWfeT 
are concluded to be behavioral ly equivalent, ( 
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As indicated, the object'of the proposed T-40 experiment ts to 
examine the methods for summarizing operator control outputs across 
.different levels of conditions in the T-40 trainer. In this experiment 
the data of interest are the operators' corvtroJ outputs f6r a specified 
period of time under the given experimental conditions after he has 
reached a specified level of performance in systems output for that 
condition, -The subject pilot will practice the condition until he has 
reached the' specified performance level ,on the system output measures 
of heading, altitude, airspeed, pitch angle and bank angle. The 
^.^^^veral conditions of the experiment are given In Table 5J. 



Table 5, I 

Experimental Condi tion^^ for Investigating 
Performance Measures in the T-40 Trainer 



CONDITION 



VARIABLES 





TURBULENCE LEVEL 


TRAINER MOTION 


MANEUVERS 


1 


Low 


Oo 


St 


& 1 eve 1 


2 , 


NVDderate 


On 


St 


& 1 eve 1 


3" 


High 


On 


St 


i level • 


4 


Low 


Off 


St 


& level 


5 


Moderate 


Off 


St 


& level 


6 


High 


Off 


St 


& level 


7 


Low 


On 


30° 


bank turn 


8 


Moderate 


On 


30° 


bank turn 


9 


High 


On 


30° 


bank turn 


10 


Low 


Off 


30° 


bank turn 


1 1 


Moderate 


^ Off 


30° 


bank turn 


12 ' 


High 


Off 


. 30° 


bank turn 



The following list of plans and procedures apply-to the conditions 
given in Tab le 5.1: * 

1. Trials are two ^minutes In length. 

2. Subject pilots are to be experienced Instructor pilots. . 

3. Subject will practice the maneuver under each condition 
--4jniU he reaches criterion performance on the^ system 

specified output measures. , . 
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4. System output measures are heading, altitude, airspeed, 
bank angle and pitch angle, 

5. Operator output" measures are the subject of the Investigation 
and analysis Is performed on the recorded output of fore-and- 
aft and lateral stick and of throttle. 

6# Conditions tabled above are for a single subject,. The order 
of trial conditions may be varied from subject to subject to 
obtain an estimate of order effect. However, the subject 
practices e^ch condition to criterion level so this effect 
- should not be significant. It is desirable to space trials 
and conditions to avoid fatigue and boredom. 

7. Subjects repeat the conditions after one day and again after 
one week. 

In this experiment the measures of particular iaterest are the pilot' 
contrt>i output measures at the time h^ has reached speeKled levels of 
performance on th^ system output measures. The control measures are 
det-ived from fore-and-aft and l4terdl movements of the stick and move- 
Tnents of the throttle. The^se movements may be summarized in a number 
of ways. ■ What is sought is a simple, on-line summarization which re- 
flects reliably the pilot*s performance and is sensitive to changes in 
it. It should reflect changes with pract}ce and level off at the same 
point at asymptote. Those measures which are proposed to be examined 
are: Stick force (mean and mean square value): stick Z score and 
throttle Z score as developed by Dr. W. Waag of HRL/FT for the ASUPT, 
program (the sum of the squared differences between present stick vector 
pos.lt ion and last stick vector position): stick fore-and-aft and 
• lateral position computed In the same tnanner as stick Z: the proportipn 
of operator 'Output power below^fe radians and above 2, termed cross-oveV 
power (see Norman, 1974): and transforms of the pi lot output power ^ 
spectrum which allow determination of the upper breakpoint frequency of 
the pilot^s output (§ee Matheny et al., I974)r It is also proposed that 
the power In narrow frequency bands around 6 radians will be examlne^d 
for the shift In power as a function of experimental conditions. The 
results of these analyses will be used to guide the development of 
operator performance records In the ASUPT and the Instrumented aircraft. 

5.2. A Operator Performance Measures - ASUPY 

It Is proposed as a p^rt of the col lection, of technological base 
Information that an Initial experiment be carried out in ASUPT. The 
principle purpose Is the gathering- of Informaftlon about operator 
performance measures and equipment procedures. However, at the same 
time-. It may provide Information and Insights about the effects of some 
relevant variables. This experiment Is proposed to repeat the 
essentials of the T-40 experiment described In Section 5.2.3 with art 
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additional variable 1|^d "vJsual display". The selection of conditions 
will be guided by the .results of "the T-40 experiment once they are 
available. However, for planning purposes, it is proposed that the 
variables and levels listed in Table 5.2 be those to be investigated. 
This experiment becomes one which, essential I y examines the question of 
the interaction between the motion and the visual display in the 
simulator. 



Table 5.2 

Experimental Conditions for Investigating Operator 
Performance Measures in ASUPT 



CONDITION 



VARIABLES 



Turbulence Platform Motion Visual Display 6-Seat 



I 

2 
3 
4 
5 
6 
7 
8 
9 
10 
1 1 
12 
13 
14 
15 
ib 



Low 

yi.gh 

Low 

High 

Low 

High 

Low 

High 

Low 

High 

Low 

High 

Low 

High 

Low 

High 



On 

On 

On 

On 

Off 

Off 

Off 

Off 
On 
'On 
On 
On 
Off 
Off 
Off 
Off 



On 
On 

Off 
Off 
On 
On 
•Off 
Off 
On 
On 
Off 
Off 
On 
On' 
Off 
Off 



Off 

Off 

Off 

Off 

Off 

Off 

Off' 

Off 

On 

On 

On 

On 

On 

On 

On 

On 



The general experimental paradigm is the same as in T-40 experiment 
in that experfencsd pilots would practice each experimental condition t 
a specified level at which time control outputs would be recorded over 
a two minute trial for analysis. Although the conditions are the same 
as those for the T-40 experiment listed in Section 5.2.3, they are re- 
peated here in the Interest- of completeness. 

1, Trials are two minutes in length. 

2, Subject pilots are to be experienced instructor pilots. 

3, Conditions are to be carried out for both Straight and 
Level and 30" Bank maneuvers. 



t 




/ 

4, Subject will practice the maneuver under each condition 
until he reaches criterion performance on the system 
specified output measures, 

ft 

5, System output measures are heading, altitude, airspeed, 
bar>k angle and pitch angle. 

6, Operator output measures are the subject of the Investigation 
and analysis is performed on the recorded output of fore- ' 
and-aft and lateral stick and of throttle. 

7, CoViditions tabled above are for a single subject. The 
order of trial conditions may be varied from subject to 
subject to obtain an estimate af order effect,- However, 
the subject practices each condition to criterion leVel 

so this effect should not be significant. It is desirable 
to space trials and conditions to avoid fatigue and boredom, 

8, * Subjects repeat the conditions after one da^ and again 
) after one week. 

The contact visual display to be used is the full width of view 
present in the ASUPT.' The content of the display. .should consist of a 
definitive horizon line with distinguishing features on the horizon as 
external referents for a heading index, A ground plane selected from 
those available in the visual display data bank judged by instructor 
pilots to be representative of the local terrain will be used, 

5.2.5 Operator Performance Measures in the Instrumented Aircraft 

The need for operator performance data obtained in th^ aircraft was 
indicated tn the discussion of methodological considerations. Section 
4,0. Operator performance output data in the aircraft being simulated 
is proposed to be used to establish a data base for comparing performance 
in the ASLIPT to that in the aircraft. Data reflecting how the operator 
responds in controlling the aircraft under representative maneuvers and 
conditions are fundamental, to developing the performance equivalence 
.approach to determining the necessary and sufficient simulator require- 
ments. By way of review, the performance equivalence approach postulates 
that two systems are equivalent with respect to performance if operator 
output performance is the same for both systems when those systems are 
being controlled to the same levels of system output. 

The existence of the ASUPT and an instrumented T-37 aircraft 
presents an unusual opportunity to develop and test more objective 
methods for stating and evaluating training simulator requirements. The 
unique capability of the ASUPT of being varied. Jn its characteristics, 
coupled with the capability for obtaining reliable measures In the air- 
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craft being simulated, makes possible research Into the perennial 
problem of how phychotoglcal equivalence can be established between the 
simulator and the vehicle being simulated In terms which are quanti- 
tative and useful, to the^ simulator design engineer. 

As Indicated In Section 4.0 it Is proposed that pHot performance 
be recorded In the instrumented T-37 aircraft across a representative 
set of maneuvers and conditions and a data base be established with re- 
spect to this performance. It Is proposed that summary measures of the 
time varying^ output to the control by the pilot be obtained and that 
these measures be gathered using highly experienced pilots flying each 
maneuver to a specified criteria. Data will be collected such that the 
population statistics of mean and variance In performance may be 
estimated. These /Statist lea I values wNI represent the population 
values for the parameters under Investigation. The operator control 
performances of interest are proposed to be the Inputs Into the stick, 
both fore-and-aft and lateral, and Into the throttle. 

A further useof the data obtained in the' Instrumented T-37 air- 
craft Is Its use In summarizing the kind of control Input behavior 
exercised by experienced Instructor pilots In the T-37. These data can 
be considered to be the criterion performances toward which the student s 
behavior Is to be shaped during the training process. Continuous 
monitoring of the trainee's control Inputs will provide diagnostic infor- 
mation which may be fed back to the student to Influence his rate of 
learning. 

In carrying out the Instrumented aircraft flights experienced 
Instructor pilots will perform to specified system output criteria during 
C6rta\n designated maneuvers. System performance measures will be re- 
corded along with the operator's control output performance. System per- 
formance measures are to be the same as those recorded in the T-40 and 
ASUPT experiments and are heading, airspeed, altitude, pitch angle and 
roll ^ngle. The records of control movements during these performances 
will be summarized In the. same manner as that developed from experimen- 
tation In the T-40 and ASUPT. The measurement parameters to be recorded 
In the aircraft with the ranges, accuracies and sample rates are given 
In Table 5.3. 

The linear accelerometers listed In Table" 5.3 are proposed to be 
mounted orthogonal to each other at a position on. the pilot's seat as 
close to the pilot's head as possible. 

The conditions under which data are to be collected are given in 
Table 5.4. This table must be Interpreted In connection with the flight 
pattern and sequence for recording trials depicted In Figures 5.2 and 
5.3 respectively. 
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Parameters 



Parameter 



Table 5.3 
Proposed to be RecorctedMn 



the Ins' rumen ted T-37 Alrcra-ft 



Elev Stick Force 
Al leron Stick Force 
Rudder Force 
— Elevator Position 
Al leron Position 
Rudder Position 
~ Throttle Position' 
Low Altitude 
High Altltudel o 
G»s ~ 
, Heading . 
Yaw Rate 

Trim Tab Position 
Pitch Rate 

« 

AI rspeed 

< 

Ro 1 1 Rate 
Roll Angle 



Pitch Angle 



+ IK 

ilVSec 
+ 1° 

+r 



Accuracy 


Sample 
Rate/Sec 


+.1 lb 


100 


±.l lb 


100 


±.1 lb 


100 


.5° 


too 


.5° 


100 


.5» 


• 

100 


.5» 


.~ioo 7 


±20' 


10 


+50' 


10 


±. IG - 


10 


±1° ' 


10 


+ lVSec 


10, 




lob 


iVSec 


iO 



100 
10 
100 
100 
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Pa rameter 

Right Eng RPM 
Time 

Event Marker 
Linear, 

Accelerometers (3) 
Angular 

Accelerometers (3) f 



Table 5«3-*(Cont inued) 
Range Accuracy 



0-1 1 0< 



Actuated by IP 



0±3g 



.Olg 



+ 2 rad/sec 4*/sec'^ 



Sample 
Rate/Sec 

10 

10 

10 

100 

100 
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Trial Order* 


A/S 


Inst /Contact 








A 


200 


Inst 


B 


200 


Inst 


A 


100 


Inst 


B 


100 


Inst 


A 


200 


Contact 


B 


200 


Contact 


A 


100 


Contact 


■ B 


100 


Contact 



Table 5»A 

Conditions and Seauencing Data Collection Sorties 
In Instrumented T-37 Aircraft Per Pilot 

Sortie 
I 

2 
3 
" 4 

5 

. 6 
7 
8 

* For trial order see Figure 5,3 

In the instrumented aircraft It Is proposed that data be collected on 
the maneuvers of climb, descent, straight and level and 30 deg turns. 
During these maneuvers tolerances will be held as closely as. possible to 
the standard determined for that aircraft. Data will be recorded during 
the periods shown In Figure 5.2. This scenario assumes that flight 
data will be collected with the aircraft being flown from Wright-Patterson 
AFB to a test flight area and that return from that area may be made at 
low altitude during which additional data on straight and level may be 
collected. It is proposed that the maneuvers be carried out both on 
instruments and contact with the contact maneuvers being flown following 
initial analysis of the instrument data. 

A detailed scenario of the data collection flights in the Instru- 
mented T-37 aircraft is given in Appendix E. The mahner in which it is 
possible to obtain a sample of sixteen two-minute trials for each of ' 
the maneuvers of straight and level and 30 deg bank for two levels of 
airspeed In counter-balanced order Is Indicated in Figure 5.3. The 
arrangement of data collection trials In Figure 5.3 Illustrates the 
order in which the two minute trial segments may be recorded for both 
straight and level and 30 deg banks during blocks of two sorties so 
that the two-minute trials for each of the maneuvers may bh collected. 
For example, in sortie one data Is collected fn order A. Return to Base 
from the test area may be flown at a 2,000 ft. altitude during which 
three more two-minute straight and, level trials may be recorded. In 
sortie 2, trials are flown In order 8, again with optional data collection. 
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Tria.l 
Sequence 

A . S&L . 30" . S&L . 30° . S&L . -SO" . S&L . 30° . 



^ S&L - 3,2 min trials (opt ional ) 
Return to Base 



. ^n" , S&L , 30° . S&L . 30° . S&L , 30° , S&L . 



^ S&'l - 3, 2 mIn trials (opt ional > 
Return to Base 



Figure 5.3 Order of trials for each block- of 2 sorties for 
the flight conditions given In Table 5.4. 
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trials recorded during return to base. Each pilot performs eight 
sorties in obtaining data on, these maneuvers, both on Instruments and 
contact, i.e., t^pse shown In Table 5,4, It Is proposed that a . . 
minimum qf five pilots fly the aircraft data collection sorties and 
repeat them In the ASUPT simulator. As indicated earlier comparison* 
of the performances in the simulator and the aircraft will allow for- 
adjusting the simulator to be equivalent In terms of pilot performarice 
to the T-37 aircraft. 

After completion of data collection under the conditions listed 
in Table 5.4 data will be collected on maneuvers during which the air- 
craft is flow^n closer to the limits of Its performance envelope. 
These maneuvers are stalls and loops. It is proposed that each of ftVe 
5 subject pilots fly 8 stalls and 8 loops maintaining as close tolerance 
to prescribed limits as possible while the parameters listed in Table 
5.3 are recorded. It is estimated that this will require two additional 
sorties. 

In summary a total of 8 sorties per pilot will be neces3ary for 
complete data coj lection on climbs, descents, straight and level and ^ 
30*^ bank turns. An additional estimated 2 sorties will, be necessary 
to obtain data on stalls and loops. 

The data should be fo,rma*tted for analysis as follows: 

1. Nine track tape. 

2. Less than 136 characters per physical record 

3. 1600 or 800 bits/ Inch 

,4. EBCDIC (Extended Binary Coded Decimal) 
5. Even parity ^ 

^2.6 ASlJPT as a Crlferi^on Qevtce 

The development of system and operator performance meastjres, both' 
in thev ASUPT and In the irtstrumented T-37 alrcf^a.ft are design«fd to 
provide objective data for establishing tlje ASUPT as a criteriort device 
fpr research. Establishing ASUPT as a criterlpn device , makes- posst,ble 
its use in Investigations of efficient training methods and the minimum 
essential training simulator requirements. The. manner In whl.ch It Js 
proposed that this be done li through the* performance equivalency* 
approach discussed In Section 4.0. , 

Performance data collected in the Instrumented aircraft provide 
the data J>ase against which the performance In^ ASUPT may be compared. 
It Is to [be emphasized that'the comparability of the two devices Is 
being esl^abllshed with respect to the closed-loofi dynamic tracking 
behavior required of the pilot in controlling the systems. The • V 
comparability of the two devices with respect to the procedures 
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whereby other piloting activities are carried out are established on 
the basis of task analyses and other direct comparison methods. 

The performance equivalence approach Is used to establish the 
equivalence of the simulator to the aircraft on the basis of Its closed- 
loop control characteristics sometimes called handling qualities. As 
indicated. in^Figure I the present method for establishing the simulator 
as being equivalent to the vehicle being simulated is the use of expert*^' 
opinion. These opinions are those of the design ^engineer and the ^ 
experienced pilot. The aim of the performance equivalence approach Is ^ 
ta provide, more objective data through direct comparison of perform- 
ance both in the aircraft and in the simulator. 

6.0 RESEARCH OUTLINES 

The discussion of suggested research is divided into that concerned* 
with training methodology and that concerned with simulator requirements. 
Both of these areas are important to improving the effectiveness of the 
training program. It is postulated, however, that the area of training 
methodology can have the most immediate and significant effect upon the 
improvement of flying training programs. The areas of Investigation listed 
in Tab'le 5.1 are discussed in this section with details of the experiments 
recommended, to be conducted. 

6.1 Training .Simulator Requirements Research 

The major areas of simulator requirements research judged by the 
panel to be hjgh priority items were those of ( I ) the visual display to 
be used In the simulator, (2) the characteristics of the motion system 
of the simulator and (3) the Interaction of the two. Any research pur- 
porting to deal solely with the visual or motion cuelng'prob lem must, 
diie to. the nature of the process, be considered as a study of the Inter- 
action between the motion and the visual areas, f^tlon-vision interaction 
studies are discussed in detail In paragraph 6.1.3. The two areas of 
motion and vision research are tak6n up separately here solely on the 
basis of the emphasis placed upon one area or the other In a given re- 
search study and does not imply a neglect of importance of the Interaction 
effect. Thus, in the research dealirig with motion, emphasis Is on the 
Investigation pf a wide spectrum of motion variables under cert^aln 
selective conditions of visual display. In the visual display resear 
area, primary emphasis has been placed upon the conte.nt of the visual 
display and its effect in various' train Ing tasks and maneuvers under 
/conditions of simulator motion. Therefore, \n reality all experiments 
proposed to be undertaken in each of the areas provide information as to 
the Interactive effect of various motion and visual display conditions. 
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6 J, I Investigation of the Content of the Visual Display 
6, 1 . 1 , 1 Background 



/ 



The problem of the content of the contact visual display In the 
simulator received the highest median rating by the Training Problem 
panel. The importance of the outside world scene in the simulator Is 
underscored by the data collected by J, Brown <1973) from Air Force UPT , 
students and instructors. These data show the traffic pattern to be 
judged t{)e most critical element of flight Instruction, and the most 
difficult to teach and to learn. A contact visual scene is necessary, 
of course, for teaching this maneuver* in the simulator. 

The question as to the appropriate features of the contact world 
which should be displayed In the simulator has not been adequately 
answered sincd the bepinhing of the use of simulators. Advances In^the 

V 5tate-of-the-art haye produced several types of Complex visual display 

generation and presentation devices. It is now possible to Investigate 
a broad range of -variables and to gain definitive information as to the 
most effective display content for the training conditions of interest. 

Attempts at identifying and describing *the visual elerftents of a 
display used by a pilot as cues to control o-^ his aircraft have produced 
both highly complex, and very simple displays. On the one hand very 
detailed displays have been produced In order to Insure that all crltjcal 
elements are Included. On the other hand, ^ome very jelemental displays 
have been used or Investigated. Both types are considered by their 
users to be useful icfr^he purposes Intended. 

Prior to the completion of the present" study, agreement had been 
reached with,HRL/FT that the content of the visual display w"as an Im- 
portant arfea of investigation. It was also agreed that during the period 
of 41e»iv6ry, Installation, and c,^ I ibratlon^of pertain research equipment 
investigations would be undertaken in this area. It Is necessary to 
narrow down the number and levels of the variables through 'as systematic 
and yalid a procedure as posslbl-e before undertaking ASUPT experimentatloj 
This is necessary since the numbg^p-^nd ccymbinatjons of .vaj?TaBtes- possl b le 
of being investigated become aarfronomical wi thomSsuch a^ screening ^process 
The use of eye movement recordings of experienced pilots fj^ylng.the u/ider- 
graduate pilot training maneuvers was proposed as a means of obtaining * 
useful information about the ol>jects and features cA the real world 
scene. used by the pilot, (see Figure 5.1) At the sa'me tjme, datb on '"T> 
the dimensions of the i\e\6 of view being used by the pilot may also be 
obtai ned. * . ^ \ * . * ' - 

Based upqn (Bynum et al 1973). experience with eye-movement, recoffl- 
^ ings av^d using the visual , model -for closed-lc^^control developed by Li fe 
Sciences, a program was out J} ned for obtaining reformation relevant t6 
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the problem of the content of the display. This program wa^ intended / . 
as one which would provide useful information upoa which to base/ASUPT 
experiments relevant to visual display content, and wht^^h could, be . ^ 
carried out^prlor to the installation of the ASUPT equipment;" /To y ^ < 
appreciate how these prel iminary ^investigations support ^SUPT experl- , , 
mentation afed to relate'the eye movement recordings to the vl.feuat\mo,d^J.,. 
a brief description of the model s'nd the part- played by eye moyerneht ''.'V" 
recordings Is in order. . • . . . 

6.1.1.2 A Visual Model as a Basis for Hypotheses . : 

The model referred to has been descri bed elsewhere InThl^lges and 
Matheny, (1971). in brief, thjs' model, assumes that external referents 
in the real world and internal refef-ents fixed to the vehicle being ; 
^ control led .may be projected through perspective geometry upon a picture 
plane perpendicular to the pi lot^s 'Mne-of-regard< ' The, relationship. ' 
between the internal referents and. the external referents provides Infpr-^ 
mation with respect to departures from de^ire^J posi tipns. of the aircraft" 
in its six. degrees of 'freedom of motion. Different positions of the|e 
referents on the picture plane will affect differentially the pilot's^, 
abilityato discriminate changes in position of the varlQU§ di,fitenslbns; of 
movemenr'of th^^^^fr^ia^l". For example, an extern^ ^rv4 interffal retererlt^ 
picked near the vertlca^l mid- line and on the horizon 6n. tha'^fjlcture^ 
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plane will not allow the pilot to di5ef=Jminate as^tine a change In bank; 
position as if those referents w^re picked further out on the horfzori r 
from the mi)f-l ine. Also, referents picked^near the ^horizon 'do not allow 
for as fine a discrimination of. forward translation as, do refereot^- 
picked closer to the airchaft on the"0arth»s>surface, rl .e., downward; r 
from the horizon line on the picture plane. ' . . 



It is a tenet of the model that it is necessary for- the. pilot to 
select internal and exterijal referents'* which allow^hlm to maKe dls- * 
criminations as to ^the changes in the attitude and position l^n s^a,ce of', 
his aircraft. It fs assumed also tha^ a p^rt of the pilots learning 
process IS the^ identi f ica*t)io'n ofthe most appropriate extfernal'and In- 
'ternalv referents fpr use in exercising closed-loop control*, tt .15, . 
.there^t^V^^T^cessary not only to identify^the appropriate referents 'f>. " ' 
tf^S^^Bai -world scene, but to describe the^ "noise" from whlqh theSe refferents^ 
Spust be &xtracte5 by the piiot. • . - / ' 

6.1.-1.3 Eye-Movement Recordings' ' ' ' ^" -v.. » ^ . 

' • ''-^ r ; • % ' \ ' " . ^ 

As rhentioney,^" experience i,n eye-Wovement recording (Bynum et aU, ' • 
1.973) led to the belief that such recordings could be.ysed' to Identity 
•'.external and Internal referents being used by ttte pilot In the 
performance of the Undergraduate Pilot Training Maneuvers. Their . ^ * 1 
identification could then lead to recommendations as the variabiles ta 

^ f • ..... 'U\ , 
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be Investigated more systemat fca I ly In the ASUPT complex. Therefore, a 
program for gathering and analysing such data vfas outlined and re- 
corrvnended, Thespian for collection and analysis of eye-movement data Is 
given In Appenditc In that Appendix both content and field of view 
•^Investigations ajKS discussed since obtaining Infor mation in the extent 
^ 'of >7isual field used by the pMof'may be oDTalned trom rnese recordings^ 
The field of view to be displayed In the simulator Is a practical 
problem about which direct information may be obtained without added 
^ . co5t^ even though the panel of judges did not rate It particularly high 
as*^ research problem. 

Experimental Approach 

It Is appropriate at this point to discuss the distinction between 
the role played .by the, performance equivalence approach to experimentation 
and the Investigation of the configuration of the visual display for 
training purposes. As Is pointed out In Section 4.0, the performance 
equivalence approach suggests that by using an highly experienced pilot 
as a> standard controller, that configuration of the visual display which 
Is equivalent to the real world* scene In terms of performance can be 
arrived. at through systematically changing physical variables In the 
^ASUPT visual display and observing performance* So long as both system 
-^nd operator performance ore- -the- same as in the aircraft the systems 
judged to ba equivalent. 

As indicated In the dlscuss.lon of estab I jshlng- a technological base, 
the first Interest Is In the minimally essential set of visual content 
conditions which are equivalent to per'formance (system and operator) In 
the 'al rcraft. -Of equal importance Is the determination of that visual 
display which Is judged by the performing pi lots Yo be acceptable as 
representative of the r;eai world scene. The visual display configuration 
will then be taken as. the criterion visual system which may be use,d^ln 
both performance equivalence and transfer experiments. The criterion 

, vlsua^l^^syjstein must be distinguished from those configurations which might 
Ibe best as aids to training and from that conr iguratlon which represents 
the absolute minimum set of elements for providing the Informatfon to 

* the pi lot and makes It possible for him to perform as he doe's. In the ^ 
aircraft. ^ ^ 

. ' With respect to training, different hypothesis as to how changes fn 
display content shpuld be Intrpduiied to produce the most effective ' 
tralnee,t)rogress may be tested sysVemfitlca Tfy using the peVformance 
equivalent system as the crTterlon system.' for example. In training the 
^udent to discriminate the visual cues for level off and touch doy^n in 
landing; hypotheses regarding the optimum number and pat+eVn^df lines ^ 
for training Jn these cues may be tested. ♦ Instructional assUts.'; 
entirely foreign to the real wor'ld scene may be.tested also to point 
\ out to the student the most r'e levant cues to his task of 'the moment*. 
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It will be recalled that In the study by Payne et aL CI954) It was 
found that the Introduction of an Instructional aid in the form of a 
"preference guage on the visual display enabled the student to learn the 
proper relationships between the approach Impact point and horizon line 
when they were unable to do so without the aid. Such training aids or 
displays may bring the student more quickly to the point at which he 
can control the system, (both with respect to system and operator out- 
put) as well as I'he'^ experienced pilot when he (The student) uses the 
training oriented display. Thus, the training system is made more 
eff relent. However, the final level of skill to be attained Is control 
of the system, with respect to both system and operator output, using 
the display configuration which was found with experienced pilots to be 
equivalent to the real world scene if the training system Is td be 
effective. 

It may be noted that at this point the assumption" may be ^de that 
transfer will be positive an'd high 'from the equivalent simulator system 
slnc^ perfonfiance of like tasks requires like operator performance. 
However, In Section 6,1,1^^ tests of the re latlonshlp^ of performance 
. equivalence methodology to classical transfer experiment are proposed 
to be conducted, 

6,r,l,5 Experimental Investigations 

The Tlrst four proposed studies of the content of the visual displays 
are shown In Figure 6,1. In brief, they comprise (d), the establishment 
of the ASUPT as a criterion system, (b) the detdrminatlon of the trans,fer 
from that system to the aircraft, and Cc) two studies which are designed 
to test the performance equlvajence approach while obtaining Information 
relevant to two different configurations of the,vl^sual display, 

In determining the ASUPT cr iterlorf system' two criteria will be used. 
These, are (U equivalent performance by experienced pilots both In terms 
of system and operator output, and (2) consensus of opinion among these 
experienced pilots that the, vlsua-l scene in ASUPT is subjectively 
acceptable as an adequate representation of the real world sceoe. The 
methods^ and measures for determining perforjwnce equlvalenc'e wqre dis- 
cussed Jn Secflon 4.0, 

^ three; major variables are proposed to be important In the'ASUPT 
ifivestlgatl-on of -the visual display, Thfese are Ca) the number of objects 
In the display; Lb) the placement or position of the objects In- the dis- 
play; and (c) the "sty! Izatlbn" or amount of detail 'in each object. 
The number and placement of objects to be tested can be more completely 
defined after analysis of the eye-movement data. However, certain 

• guidance for hypothesis formujatlon can come from the visual model 
referenced earlier. This model woul dD-suggest that veil ddfined ex- 
terna*! reference points -appropriate I y placed will provide Information 
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to the pilot for controt of the six ill mentions of movemQnt of his air- 
craft* Certain positions of the referents are rnore appropriate to 
giyen dimensions of movement than are others. 

For control of the three attitude dimensions, pitch cdntrdX^e- 
quires a reference object directly ahead of the aircraft on^the vertical 
mid-line and as near to the horizon as possible, For'control of this 
dimension alone, only an Identifiable point or object Is required. This 
obtains as a general rule for the control of^pltch across all of the 

'fnaneuvers to be flown. However, for each particular maneuver the place- 
ment of the reference object with respect. to- The ground or sky plane Is 
specific to that maneuver in order for It to be accomplished bes,t. For 
straight and level flight a pitch reference point ahead of the aircraft 
at the level of the horizon line, or the horizon line alone, will provide 

'a reference against which the pilot carv judge some Internal referent on 
his aircraft InT order to hold his pitch attitude. For a nose down 
attitude during approach to landing a reference object lower on the 
grcund plane would be hypothesized to be preferable. 

For bank control, objects displaced laterally from the longitudinal 
center line of the aircraft provide greater amounts of positional dis- 
placement per unit of bank as they move further out from the center line. 
Thus, an object placed on the horizon at the vertical mid-line would 
exhfbit little perceptible movement to the pilot per unit of bank. That 
same object placed 30 deg In azimuth from the vertical mid-line would 
provide a greater amount of displacement for the same amount of aircraft 
bank and thus enable greater precision of control, " 

With' respect to heading control, objects on the hopjzort^' In the 
forward viewing area to which the pilot can relate an Internal referent 
will provide the S2|me amount of displacement on the display per unit of 
heading change. However, for ease, oIp scan for the pilot the object should 
be directly forward on the vertfcal mld-lfne and on the horizon. 

For the detection of longitudinal motion of the aircraft objects 
directly In front of the aircraft which appear to move toward It are 
necessary for the best detection of this movement. Objects -dl recti y 
below the aircraft have a greater perceived displacement while those 
direct ly forward and on the horizon have the least per unit of longitu- 
dinal movement. Therefore, to max'imize the pilpt^s detection of forward 
motion or change In that motion objects as nearly beneath the aircraft 
as possible provide the^ greatest Information for control of thlis" 
dlmeoslon, .* 

, » » \ 

The control of lateral displacement parallels that of longitudinal* 
For dfscrimlnatioa of 'displacement along the vertical axis the pilot 
rJust discriminate changes in the size of objects In that he must dis- 
criminate changes In' the relative distance between one edge of an object 
and another 'or the chahge In distance between objects. 
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The cues Just discussed are those which allow the pi lot to control 
the attitude of his, aircraft about Its three axes and to discriminate 
whether tt has changed Its position In three dimensional space. In 
order to successfuHy fly his aircraft, however, he must discriminate 
and Identify objects In the real jworld which allow him to 'idl rect^ his 
aircraft purposefully from point to point or In the fulfillment of 
^ome objective or mission. Those objects which he must Identify and 
use for direction of his vehicle are unl<jue to his purposes and, object! ve 
of the moment. Thus, when the objective of the pilot Is to land the 
aircraft on. a particular spot the characteristics of that spot necessary 
for him to Identify It must be present In the display. Similarly, he 
may be controlling his aircraft In three dimensional space to keep It 
positioned upon another aircraft during formation flight. The relevan.t 
cues peculiar to the other aircraft which allow him to maintain his 
position must be presented In the vlsuaTdlsplay in order for the 
trainee to practice In th^lr use and becoroe proficient In formation 
flying. 

It Is the overall goal of research In simulator vlsual^^^^isplays to 
Identify those essential cues which allow the pilot to control the 
attitude of this aircraft and to position It In three -dimensional space 
In accordance with the objectives and goals of the moment. Further, It 
Is the objective of that research to determine what assists or Instruc- 
tional aids may be added to the visual display to enhance the student's 
attamment of proficiency In the use of these cues. It Is the function 
of'maneuvers In the pilot training syljabus to pra^rtlce the student In 
the accurate attitude and positioning control of his aircraft. Certain 
maneuvers have specific and universal objectives such as landing the 
aircraft. However, In the main," maneuvers are not practiced for ^ 
maneuvers' sake but rather to practice the student In control of the ^ 
various dimensions of movement of the aircraft over which the student 
' must have control In order to accomplish specific objectives and pur- 
poses. Therefore, it Is suggested that the'vlsual display for the 
^ beginning student must b6 composed of 'that essential set of features 
which allows him to exercise control about the aircraft axes and along 
three dimensional space axes as a general set of ski Ms. He must then 
' learn to extract from the real world scene objects which serve him In 
carrying out his control and navigational functions. ' He must learn 
also to discriminate these referents when they are burled in visual 
"noise". The term visual noise Is used here to mean conditions In the 
real world environment which tend to obscur^.or make difficult to dis- 
criminate the referents which the pilot wishes to use. Atmospheric 
attenuation, smoke, haze or any condition which causes an object to have 
low definition constitutes such noise. To the extent that this noise 
degrades the pi lot's discrimination of changes between the referents 
which he Is using to control his aircraft his precision of control must 
be- degraded. 



ERIC 



V 



_ . The analysis of the eye-movement records will provide relevant data 
for Identifying the characteristics of those objects which must be 
Identified to attain the goals of the pilot and to Identify those 
factors In the environment which tend to Inject noise Into the visual 
scen^. The objects which must appear In the visual di$play to allow him 
to exercise attitude and position control are not of necessity local 
terrain features and could be quite abstract, How.e_ver, it will add 
appreciably to the realism and, therefore, the acceptance of the display 
If they are recognizable as local features. 

The Identification and dlscriminaf ion of changes among objects may 
be hypothesized to be a function of the level of detail of fhe object 
and Its contrast with the background. These factors are controllable 
as experimental variables In ASUPT and may be varied systematically 
through the vse of the number of lines used to define an object and the 
shades of gray used to provide contrast between objects or between ob- 
- jects and background (figure-ground contrast), 

Injthe light of the above discussion the first experiments dealing 
with the content of the visual display are divided Into the two cat- 
egories of (I) that content necessary for control of the attitude and 
position of the aircraft In three dimensional space, and (2) those 
relevant to .the Identification and use of features of the environment 
which allow the pijot to* controJ his vehicle In attainment of some 
specifiable goal or purpose. Results from the first of these will be • 
appropriate to the specification of displays for simple basic aircraft 
control trainers. The results from both provide Information for specifi- 
cation of more complete mission: oriented trainers. The variables and 
their levels for studying attitude and position control are given In 
Tab le 6. I • t 

This' experiment Illustrates the necessity for establishing the 
dfscrlmlnable Intervals of a variable before going Into the experiment 
proper. Jn the choice^ the^shades of gray for any object It Is 
evident that If shade of gray Is to be varied over several levels those 
levels must be chosen which are discrlmlnably different by the subject. 
Should any two levels chosen not be discrlmlnab le it follows that they 
actually represent one level of the variable to him. 

The discrlmlnable levels of gray mgy be established quickly simply 
by asking the subject to fly the display while observing the shades 
of gray In specified objects. The shades of gray In these objects are 
then varied up and down the scale with the subject reporting each time 
he discriminates a change In shade. Jhls simple psychophysical procedure 
will serve to establish those Intervals minimally perceptible by the 
subject while he Is engaged In his flying task. Establishing these 
Intervals allows the selection of levels of the variable demonstrably 
different to the subject for use In transfer of training studies. 

. 38 . , • 
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\ The'second major category of stuidy Is the determTnat Ion of those 
features of the environment which allow the pilot to change his position 
or behave relative to those features in some purposeful way. The two 
major tasks In UPT in this category are (I) traffic pattern, approach 
and landing, and (2) formation flying. Of these two training problems 
that of traffic pattern and approach to landing has been researched 
to a greater extent than that of formation flying* However, recent 
experiments at HRL/FT using the Formation Flight Trainer (FFT) has 
provided directly applicable data to the specification of trainers for 
training In this maneuver. 

The ASUPT visual display capability allows the study of the re- 
levant referents to be presented for the touchdown phase of the landing 
and how they may be encoded In the display. Satisfactory study of this 
area has not. been possible. up tp this time. The ASUPT capability makes 
it possib le. • , / ' 

• The variables and suggestions for their encoding for the approach 
and landing study are gt^ep in Table 6.2. It is proposed that the 
essential features listed under Conditions I and 2 In both Tables 2 an^J 
3 may* be better defined ^fter analysis of the eye-movement records and 
the full capability of the ASUPT for providing display features has 
been comp lete ly .checked out. 

The procedure for establishing the ASUPT criterion system In 
Study t has be^n dUcussed in Section 4.Q|Nfeth9dologi cal Considerations. 
Study 2 wi J I employ the Classical transfer of training paradigm dis- 
cussed by Gagne, Foster and Crowley (1948). 

. , Studi.es. 3 arid 4 are proposed as tests of the ^performance equiva- 
lence approbch as well as to ^assess the transfer to the aircraft of 
less complete visual systems. The paramete^s of number, position and 
styllzatlon of 'objects will be' varied so that (1) a condition not equi- 
valent to the criterion ASUPT Is obtained^ and (2) a condition equivalent 
to the crlt^riorf ASUPT system Is obtained.^ This Is to be done in the 
manneV,. described In Section 4.0 using' the performance of experienced 
pilot's*. Cfassical transfer of training experiments are then proposed' 
to b^ conducted using Conditions I and 2 of these studies to determine 
wHet'her the^^ hypothesis that equivalent systems bflng about the same 
a/noi/nt' of transfer while non-equivalent systems do not. The amount of 
transfer-obtained earlier in Study 2. 

The general design format for Studies 3 and 4 Is shown In Fig\jre 
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6,|,Z Investigations of Motion In the Simulator 
6. I .2. I General 

The problem of determining the necessary degrees of freedom of 
cockpit motion In training simulators was sixth on the priority list , ' 
of training problems, (See Appendix D) Manipulation of^this variable' 
In experimental Investigations .Is particularly easy using the ASUPT, 
'This device also has addltionaT areas of Interest to trainer motion 
design - that of gravity alignment and the G-seat, Gravity alignment 
cueing Is designed to come from tilting the platform such that normal 
gravity Is used to substitute for the acceleration or ,decleratlon of ^ , 
the aircraft particularity during take-off or landing G-seat cueing Is 
of two forms - those which come from movement of the elements of the , 
seat to provide movement cues and those which provide sustained 
pressures to the student^s body. See Appendix B for more complete 
definltlohs, 

■ 

. As pointed out by Smith (1972) it is manifestly not possl'ble or 
sensible to attempt to Investigate all of the possible combinations and 
permutations of the six degrees of freedom of motion, gravity alignment 
and the ways In which the G-seat may be used% A sorting out on the 
basis of the characteristics and dynamics of the physical system being 
^simulated and of the human sensory system was felt necessary and has been 
' done. Further screening of these variables Is proposed to be carried 
■ out In protest Ijvy^st legations as Indicated In Figure 5,1, * 

The combinations of conditions selected for recommend'ed study and 
•Ilste.d in Table 6,3 may bemused either in classical transfer experiments 
or they may be approached through the performance equivalence method. 
In- the discussion to follow the performance equivalence method Is used 
although the fact of the, previous statement must be kept In mind, 

* * ' ' • > ^ * / ^ 

*• -6,1,2,2 Experimental 'Studied 
« • ■• ' 

The specific experimental -question to be addressed by these Investi- 
gations Is "under What method of Introducing motion cues and across what 
conditIons*of flight are the, control performances of experienced pilots 
.^qulv^aJent?"' The- approach taken supposes that I f two systems are 
measurably d>ffQrent bv some quantitative metric but their operation 
resut^-s In no n)easurabie difference rn t+ie performance of experienced 
> pilots then th^ physlcerr<jrf ferences are behavlorally equivalent and are 
inot .different 'for purposes oif a training simulator. This Is the method- 
ofpgy discussed' in .Section 4,0, 

' TKe li:)dependent varl^able of prime Interest Is that of the motion cue 
(Condition, What Is sought Is a determination of whether pilot p^rform- 
"^ance varies 'as -a functl(3[n of motion cue conditions across different 
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maneuvers and for dlffereht levels of external disturbances. The 
motion cues to be investigated fall^ under the thre^ broad* categories 
jxist .dlspussed:Ca) platform motion, tb) gravity alignment and (c) 
G-seat, Particular combinations , of .sub^^ategorles of these major areas 
have beert selected to obtain data ^about ten specific questions. These 
.conditions are listed In Table 6^3, The specific experimental questlSns 
are given In Table 6,4. 

Table 6,4 shows' the experlrrtental conditions relevant to providing 
Information about particular experimental questions, The>.experl mental 
procedure Is one In which the twenty-two conditions may be run In three 
experimental sessions and provide Information relevant to the ten 
experimental questions posed. The experimental questions have been 
selected- such that successive data runs can be made with experimental 
conditions on subsequent runs being dependent upon the findings of earlier 
runs. A" "sorting o^jt" procedure Is used In which the various combin- 
ations of coTKjftrons are examined to determine whether they effect any 
differences In pilot performance and what the relative contribution of 
i'he conditions, are to performance variability. The effects found under 
one set of conditions are used to guide the sele Atlo n of the experimental 
conditions to be used in the next run. In order to accomplish th^ls It 
]s proposed that the incremental design outlined by Demaree In "A ^ 
Recommended Design for Experimental Studies Using ASUPT" be used, ThJ/S 
, paper Is given |n Appendix H, , • - 

Under the proposed experimental procedure as outlined In Table 6,4, 
the first experlmenta'l question would Incorporate experimental Conditions. 
I, 10, II, 15, 16, and 17, Results from performance under these conditions 
are Intended to show whether or not the major conditions of simulation 
of Ca) no motion, (b) six degree of freedom platform, (c) six degree 
of freedom platform with gravity alignment, (d) slx'degree of freedom 
platform'with full G-seat, (e) six degree of 'freedom platform with full 
G-seat and gravity alignment, and (f) full 6-seat only, differentially 
affect performance and. If so, what the relative effects are. 

However, as a test of the performance equivalence approach It Is 
proposed that Conditions I (no-motion), 9 (full motion), and that' 
condition .found to be equivalent to full mptlon be used In the paradigm 
shown In Figure 6,3^ 

Irv this paradigm equivalence and non-equivalence Is established by 
the methods outlined In Section 4,0, In Figure 6,3 the dondltlon of 
no-motion Is assumed to be non-equivalent' to the ASUPT criterion system 
In terms of performance. It Is also assumed that a cpndltion of less, 
than the full criterion ASUPT system may be established. 

If Experlrpental Coadltlons I and 10 differentially affect performance 
Session Two will be concerned, with obtaining data on Experimental 
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Table 6;3 



Experimental Conditions Selected for Test in Determining 
Platform Motion and G-»Seat Operation * ^ 



Experimental 
Condition 



Condition 
Description* 



• I. 

2. 
3. 
4. 
5. 
6. 
7. 

9. 
10. 
II. 

12. 
13. 
14. 

15. 



16. • 

17. 
18. 
1^. 
20. 
21. 
22. 



* Legend; 

, P - Pitch 
R - Rol I 
Y - Y3W 
H - Heav6 



No Motion ' ^ 

P,R (Platform) 

P, R, Y (Platform) 

P, R, Y, H '(Platform) 

P, R. H tPlatform) 

P, R, H, L (Plattorm) 

P/ R, H, F & A (Platform), 

P, R, H, L, F i A (Platform) 

P, R, Y, H, L (Pfatform) 

6 Degree o'f Freedom Platform 

6 Degree of Freedom Platform wlth'Gravlty 

Alignment 

P, R '(Platform) with Gravity Alignment 
P, R, H, (Platform) with-Gcgy^ty Alignment 
P, R, Y, H, L (Platform) with Gravity 
A I ignment 

'6 Degree of Freedom Platform with Full ' 
G-seat 

6 Degree of Freedom Platform with Ful I 

G-seat and Gravity Alignment 

Ful I G-seat only 

P, R (G-seat), H (Platform) 

P, R (Platform(, H (G-seat) 

P, R (G-seat) 

P, R, F & A (Platform) 

H only 



L - Lateral 
FiA - Fore-and-aft 
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. Conditions 2, .3, 4, 5, 6, 7; 8, 9, and 21. The results from these 
conditions are Intended to determine what combination of degrees of 
freedom of motion 5f the platform a re* equivalent* 

* X ^ 

Experimental question No* 3 Is answered by comparison of Conditions 
id e^nd 1 1. with experlinental run L This question concerns the contrl- 
^ butlon that the gravity aNgnment cue makes to the maximum degree of 
freedom platform. 

In Experimental Session 3 the six remaining experimental conditions 
, of 12, 13, 14, 18, 19, and 20 are added. Comparison of Experimental 
Condition 13 with Condition 5 Is Intended to answer the question with 
respect to a contribution of gravity aMgnment glven'a nominal platform. 
What 'constitutes a nominal* platform has been assumed to be that re- 
presented by Condition*?, I.e., one with pitch, roll and heave. How- 
ever, the results of the anajysis of Experimental question 2 may change 
what Is considered to be a nominal platform and, therefore, change these 
conditions sortiewhat. 

Comparison of Conditions 15" and 16 Is designed to give Information ^ 
about the contribution of gravity alignment when the maximum platform 
Is used with the G-seat. 

Addition of Condition 12 allows the comparison between Condition 2 
and M and gives Information about the contribution of gravity alignment 
with the minimum platform. I.e., one which provides only pitch and roll* 
* st I mu 1 1 . 

The addition of Condition 14 allows fqr a comparison of that 
condition wlthf;Condit ions 9 and !0 and gives Information about the 
longitudinal ac'celeratlon, cue contribution when a maximum platform Is^ 
. usod, i^e., whether the aravlty ^a I Ignment or fore-and-aft translation 
Is the better clue or whether It has any effect at all upon performance. 

In carrying out the experimental runs It Is proposed that the ex- - 
perlmenf^l conditions be carried out under Instrument fll^ght conditions 
using the maneuvers of straight and leval fllght^^ 30** bank turns and 
unusual attitudes. It Is also proposed that the fundamental nature of 
the control task required of the pilot In performing 'these tasks Is a 

\ function of the external, forcing function or turbulence acting upon 
th6 system. His control behavior will be affected by the nature of 
this disturbing function and It Is, therefore, proposed that three 

• leve^ls of disturbance be imposed upon each of the basic flight tasks In 
order to determine tjjie equivalence of various physical systems acrpss 
representative tasks and disturbances. The level of disturbance for 
each conditio^ Is to be determined through pre-testlng of the 
turbulence generation- system of the ASUPT. 

^ • ' , 48 ' 
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6.L3 ^>totlon'^»Vislon Iryteractlon Studies 

6. 1 .3. 1 General / , • ■ 

The results of certain experiments conducted In the visual and ,1* 
motion areas are proposed to be used In experiments designed to provide 
Information about the Interactive effects of these two major var'lables. 
These experiments are proposed to be conducted using +he two fpajor 
areas of visual cueing discussed In Section 6.1,1. These are aircraft 
attitude control and aircraft position control. Within these two areas 
experiments are destgned to determine whether differential Interactive 
effects occur. 

6. 1 .3.2 Exoer I mental Studies 

As mentioned, the Interactive effects af motion and vision are 
proposed to be Investigated In, two rather different areas o^ pilot t^sks. 
First, are those conditions In which the pilot^s task Is control of the * 
attitude of his aircraft. This control Is primary and antecedent to 
control of the position of the aircraft In three dimensional space. 

It Is hypothesized that the Interactive effect Is influenced by 
numerous variables In the total task situation. Primary among these 
variables are level of preclslpn and control required an3 the nature of 
the external forcing functions (ordinarily termed turbulence) Imposed 
upon the system. 

The rnodel of control behavior adopted In considering aircraft' 
attitude control Is that of the Effective Time Constant (t^X of the 
system. (Matheny A Norman, 1968) This model Is used as a basis for under- 
standing the interactive effects as well &s to form predictive hypotheses 
to be tested. In brief, the model asscnji!&^ that the precision of closed-loop 
error-nulling behavior Is a function of the Immediacy of feedback to the 
control I lag operator, e.g., the pMbt. The time taken for feedback to occur 
has be^n termed the Effective Time Constant of the man/machine System. The 
value of +^ 'depends upon the values of given properties of the machine dynamics 
and of the human controller. The properties of the human control ler of 
importafice are the fodalltles through Which the information Is received 
and the threshold level of those modalities. 

The relevance of this model to the mot lon/vlsloh Interact-Ion problem 
Is that certain Information abogt the status of the aircraft, partlcu- 
larly,,changes 'in attitude the aircraft, are transmlHed more Immedi- 
ately as feedback to the Operator through the" motion senses than the 
visual, ft Is a tenet of the model i-hat the motion sensed are cued by. 
rates of onset of accelerat loa which allow for the Initiation^ of re- 
sponse .much in advance of that which would be triggered by the resulting 
positional change of the stimulus detected by the visual segse. It 
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. follows from the model that Increasing the gain on a given dispjay to 
a particular sense will Increase the Jmmedlacy of the feedback to the 
operator. Thus, Increasing the gain of the visual display will result 
in an increase In the precision of control. Likewise, visual displays 
• which provide the operator with rate, acceleration or onset of accel- 
eVatton Information ,al low him to receive more Immediate Information as 
to the status of the system relative to the desired state and to control 

. it more precisely. ^ , 

• From the Effective Time Constant model the prediction can made 
that In systems such as the T-37 aircraft, performance of the precise 
attitude contrpj task will be enhanced by addition of the proper motion' 
cues while other tasks such as positional control In which the visual 
feedback Is timely and adequate will not" benefit greatly from the 
addition of motion cues. It can be further predicted that the visual 
display which Incorporates 'the higher gain (e.g., large contact display) 
will result in a higher precision of control. It can also be hypothesized 
th»t training a student to a given level of proficiency without those motion 
cues which will be 4> resent In the transfer tasks constitutes over training 
hlm^and he will do very wfell. I.e., transfer wl I I be high, when he Is 
transferred Into the task In which the total set of stimulus cues are avail- 
able. 

As mentioned earlier the first interaction experiments dre designed 
^ tp examine the sitbation In which interaction effect Is likely to be . 
greatest as predicted by the , effective time constant modeU This area 
is in that of close precision attitude control. The experimental, conditions 
suggested for this experljnent are given in Table 6.5. 

It Is proposed that the performance equivalence paradigm be used 
for this experiment. Experienced pilots will fly ^ach of the conditions 
and tasks to determine whether or not their output control behavior Is 
different under the various conditions given standard system output 
performance requirements. For those conditions found to l)e equivalent 
and those found not to be. It Is proposed that groups of the students 
be trained under, each condition and subseauently trGr.3ferred, to the air- 
craft as a further test of the performance equivalence approach. 
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Table 6,5 

Experimental Conditions for Initial Motion-Vision 
Interaction Experiments 



Tasks: Approach to landing. 

Straight and level. 

Variables: Motion - no motion and full, 6 degree; 

Turbulence - high and low. 
Precision of^ontrol - high and low. 

Subjects: Experienced pilots for establishing 

performance eaul valence. Trainees for 
training on selected conditions and 
transfer to T-37 aircraft. 
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6.2 Training Methodology Research 

As Indicated In the Introductory . section, this area of research Is 
proposed to have highf potential for producing results which will greatly 
Increase draining effectiveness and has Immediate application. The, 
adoption of ASUPT as a criterion system for investigating efficient 
training methods Is the basis' for this oroposaL This Is particularly 
true of research in two areas judged to be Important by the training 
problems panel; namely, cognitive pre-tralning and feedback. These two 
areas will be discussed first followed by recommendations regarding the 
^reas of sequencing of training tasks, contextua I training and Individual- 
ized instruction, 

6,2J Cognltl ve'-p re-training 
* 

Cognitive pre-training refers to th^ complete understanding of the 
cognitive and action aspects required for completion of a task be»fore that 
task Is attempted In the simulator or aircraft. In practice It may be 
achieved in several different ways. 

For the perceptual and cognitive aspects of ta?T<s,_§uch as learning 
to scan, read and^ Interpret the Instruments for Instrument flight or 
learning procedural sequences, a number of Instructional aids such as 
photos, work books^ or audio-visual devices may be hypothesized to provide 
effective training in these task-aspects. Cognitive understand I r>g or 
mental pre-practice may be highly effective pre-trainlng for continuous 
control tasks through use of simple or even no equipment at all. 

It Is proposed that the evidence Is sufficient to support Institution 
of certain forms of pre-cogniti ve training directly Into the training 
program. These Involve those techniques demonstrated to be efficient by 
Flexmarv^ et al. (1950) and Flexman, et al. (1954). In general these 
techniques requ'ire the student to become proficient in his ability to 
verbalize the pertinent cues and responses necessary to meet the require- 
ments of the task. In this verbalization It is Important that the 
Instructor determine whether the "student Is merely "parroting" the words 
of the instructor or whether he completely and thproughly understands the 
perceptions and respo nses that are required for successfully accomplishing 
the maneuver. With the advent of Increasing Initial and maintenance 
costs of simulators, the Investigation of the amount of time which might 
b% saved In reaching criteria In the ASUPT simulator for certain maneuvers 
through the use of other pre-cognltl ve training methods ts highly 
recommended. 

The Human Resources Laboratory/Flying Training Division has as a 
part of IJ:s_7?sea«ii_capabl I Ity an audio-visual Instrument training 
devlce~(AVIT) with a program suitable for pre-cogn Itl ve training in 
scanning, reading and interpreting the basic Instruments for basic 
instrument flight. The contribution of this device to reduction of 
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simulator training time for learning the basic flight maneuvers capable of 
being taught by the device should be Investigated. The device Is con- 
figured to teach a student to scan, read and Interpret his . Instruments for 
straight and level flight, 50^ bank turns and unusual attltudas* A 
functional description of the device Is given In Appendix G. 

It Is proposed that an experimental group be taught to criterion 
level In the AVIT devrce ahd compared to a control group taught only In 
the simulator In terms of time to reach proficiency In the simulator. It 
Is proposed that both the experimental and the control groups then be ^ 
taught al! other maneuvers ln*the basic Instrument f I Ight curriculum to 
determine the degree of generalization from the three maneuvers taught In 
the AVIT to, other maneuvers performed Jn the simulator/ ^ 
\ 

It IS proposed also that the capability for a device such as the 
AVIT be Investigated for Its use In teaching the perceptions required In 
contact maneuvers such as the approach to landing. This hypothesizes 
that the major portion of the learning task of the pi lot trainee Is the - 
learning of the pei^ceptual and cognitive aspects of the task as opposed to 
the motor responses he rs required to make. It Is hypothesized that once 
the pilot has learned to perceive the desired stimulus relat lohshlps, to 
recognize departures from these desired relationships .and to know In which 
direction he should Initiate control movements In order to correct them, ^ 
hls^tralnlng time In the. actual performance of the continuous time varying 
task will be greatly reduced. 

In brief ^ it is*' recommended that the use of a programmed branching 
audio-visual Instrument trainer be Investigated fqr teaching the perceptual 
and cognitive side of contact tasks since these tasks are among the most 
demanding of the trainee and require some of the most complex equipment 
for simulation. \ 

6«2.2 Feedback . ' 

in general Zeitgeist has been th^t feedback or knowledge of results 
facilitates training, (see Smode, -1958) While this Is true in the 
general case there are cautions which should be observed "when applying the 
feedback principle to the particular case or when carrying oyt research 
with particular tasks. For example, Briags (19^2) points to the Inter- 
active effect of type of a.ugmented feedback (positive or negative), com- 
plexity of the feedback criteria and level of training. He draws attention . 
also to the Importance of the feedback withdrawal schedule. Ward and 
Senders (1966) suggest that adding an element to the task such as a 
feedback method has a degree of workload' associated with It and may Interfere 
with carrying out the primary tasks due to the time sharing requirements. 

The work of Feuerzeig (1971) In providing Instructional monitoring 
is believed to be a method of feedback which should prove highly 
beneficial In Undergraduate Pilot Training but must be Investigated as 
to the Interference effects and the feedback withdrawal schedule. 
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Feuerzeig provided the trainee with cornputer generated diagnostic Infor- 
mation, instructional suggestions and a two-dimensional dynamic display 
of the 'progress through a task such as a holdjng pattern. His results 
suggest that such an approach to feedback to the UPT trainee, particularly 
for i-nstrument flight training, .should^be Invest igated^ 

*6*2«3 Sequencing of Training Tasks 

WhMe sequencing of training 'tasks was^llsted as being one of. the 
top priority items by the Training Problem P&nel, it Is suggested that ^ * 
the splution to this problem Is more properly carried out using the !SD 
task analysis approach suggested in Manual 50-2. How best tp'te^ach each 
of the separate tasks may be approached through research in training 
methodology such as those of cognitive pre-trainl'ng and feedback. .It .is 
recommended that the proper sequencing of these tasks,' however, may more 
properly be done through synlrhesizlhg the tasks into an^oVerall curriculum 
by the ,use of ISD procedures. The comparative test of two optimized 
* sequences may be carried out as a final test usfng the ASUPT as the 
criterion device, within which. all training is given and final training Is 
tested. This final cpmparative experiment should be undertaken after 
thorough ana*lyses of the tasks ,requi red of the pilot and should properly 
include a synthesis of the tasks of the total curriculum based upon the 
analysis carried out by Meyer et aL (1974). ^ ^ 

6.2.4 Contextua I Tra i n i ng 

The arguments tor investigation of contextual training are Identical 
to those just (>resented for. sequencing of the training tasks. The context 
within which certain tasks are to be trained should be established and 
synthesized by task analysis prior to any experlmenta I investigations. 
Comparative studies of two or three of the candidates for "contextual" 
training should then be investigated In ASUPT u^lng it as both the training 
and the criterion device. 

6.2.5 Individual ized Instruction 
Individualized Instruction Is such a broad and diversified area^^^t. 



ft is not possible to describe research relevant to It In this report 
The research outlined with respect to training methodologies such as 
feedback and cognitive training will provide information as to 
instructional strategies which mdy go Into an individualized Instructional 
system. It is recommended that the Indlvlduaf characteristics such as 
biographical data and subject source. I.e., 'Air Force Academy, ROTA and 
so forth be maintained on all trainees used as subject In the experiment. 
The predictive vali'dity of research findings will rest In large part on 
information about present and future trainee ,popuJ at I oris^ Thus^ a datg 
bas^ may be developed for.making management decisions about Individual-, 
ized instructlona I programs which may be automated to assign |hs^/uct1ona I 
strategies, incentives and guide the trainees* progress as a funttlon of. 
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his r^te. of ski 1 1 attai nmen t^ biographjca j 6a?J<ground ^nd t;^thQr In- 
d^v^dual characteristics upofi which he may have been*-se1ect6d. ^The re- 
search b^in^ carried oiKt by Human Resources. Laboratory, Technical Training 
Di vf si.oh, wfth respect to IndJviduaM ized trafning.wlJ I provide useful data 
for manageroent decisions about Individual 1 zed J ostrt^ct Ion In flying 
training and'shoyld.fom a part of the data-base, ' . » 
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APPENDIX A 

Letter of Invitation aruJ Instructions to 

f * Researc|i Prjoblems Panel 

* * * 

< ^ * 

Life Sciences, Inc., under Contract F4I609-73-C-0038 wlth^the 
United States Air Forced Is carrying out an investigation the purpose of 

^which is to'produce a Handbook of experiments dealing with the design 
and 'uti I ization of training ^lmulatqrs». this document will outline the 
more important experiments necessary of being carried out In order to 
answer critical questions regarding, simulator hardware design and methods 

• bf slmuJator utilization. The research to be recommended In this Hand-^ 
book is to be carried x>ut usirig'the Advanced Simulator for Undej-graduate 
P'iJot Training (ASUPT) orother research 'faci I i ties available fo the Air 
Force Human Fte'search Laboratory /F I ight Training at Williams Air Force 
Base. - ' . . 

It is expected that the problems to be attacked will be those im- 
^ porjant to a wide range of training si tuatlons '1 n whiA the trainee , 

acquires cognitive ana psychomotor ski Ms in learning to control a man- 
, machine systems ^However, our emphasis is upon the research through use 

of the qround-based trainer and training technology In beginning pilot ' 

training. 

%* 

In-order to tnake the Handbook of most value to Air Force research 
.-and to ensure the most* general ity across a wide range of beginning 
pi lot^training programs, we are attempting to involve as many Individuals 
with experience and know ledgp in the problems of beginning training as 
possible.^ >This involvement takes tKe form of asking the Individuals 
to present their views- as to the most critical training prolyl etns to be 
attacked using a facility such- as ASUPl and the other HRL/FT devices. 
Therefore, we arc attempting to obtain the considered judgments of a 
group qf experts in the field as. to the important training problems. 
After these have been assembled they will be listed and submitted back 
.to the grouD for priority judgments or order in which they should be taken 
up jji a research program. ^ > ' 

I would appreciate very much your becoming a member of this group. 
Your participation as a member wouldbe most valuable. 

I Should poi.nt out that other equipment available for research on 
training problems consists of a T-4G simulator wf^Tch Ts a modification of 
the ME-I bui.4t by Li-nk. Its main features are a mostlon base and 3 
vlsuaTsystem. The motion system provides three degrees of, motion In 
pitch, bank* and heave. The visual <li5play provides a visual Image of 
44"* by'28'^ in^full color through use of coior film and 35 mm slides. 
This visual system is known as' the Electronic Perspe,ctlve Transformation 
(EPT) system and presents an approach, landing and take*-off sequence 
filmed at William^ Air Force Base., 
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Other equfpment includes thre6 T-40 trainers which were designed 
to train flight personnel in. the typical twin engine Jet aircraft. This- 
trainer donsfcsts of a cockpit mounted on a two degree of freedom motion 
system and an instructor's station. The motion system provides motion 
in the pitch and tqI I axe§. However, the pitch motion also provides 
vertical translation at the pilot's center of gravity to simulate heaVS* 
motion. The trainer may be considered as an instrument flight trainer 
and useful' in research on the problems of instrument flight, engine 
systems, navigati^on, radio, oommun i cat ions and so forth for this type 
of aircraft. 

A third trainer is termed the simpllffed Forma-^^F light Trainer 
(PFT). This device is configured to train the studenT*in formation f lying 
of the T-38 ajrcraft. The student "flys" a television camera which views 
a, model of a T-38 training aircraft. The image of this model is pro- 
jected on a wide angle screen at the student's station. All basic formati 
tasks from join up to position keeping and in cross-under can' be 
accomplished in the trainer. 

I hope that you will be able to help as a member of our panel of 
experts and can send me your thoughts on what you regard to be the 
critical research issue in simulator design and utilization at an early 

date. • ' 

* 

Best regards. 



W. G. Matheny' 

President & Technical Director 
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APPENDIX B 

Life Sciences, 'Inc. 
Contract F4 I609-73-C-0038 

graining Problems List 

Visual Simulation Requirements 

The visual system in aircraft simulators presents the real K^^ld 
visual scene outside the aircraft to the trainee, Major problems In this 
area arej 

o Field of View - The solid visual angle to be covered by the 
disp-lay, f4easured from a forward I ine-of-regard, the vertical 
and horizontal visual angles to be covered by the display. 

o Content of the Display - Objects, points, lines and so forth 
in the dfsplay as sources of information for t^he trainee in 
carrying out his ai rcraft -control tasks, \ 

o Quality- of the Display - Aspects of the disolay such as 

brightness, resolution, sharpness, contrast, and distortion 
or aberrations of the displayed image which have to do with 
its legibility. 

o Color - The chromaticlty of the display. I.e., whether the 
visual display is achromatic or contains an approximation to 
real world color, 

o Depth of Field*- The Image being displayed ^o the trainee Is 
collimated and thus appear at^ virtual infinity or appear as 
a flat plane projection. 

Motion Cue Simulation 



. This area of research may be subdivided Into what may be termed the 
Onset or true motfpn cues and their washout and those cues which are 
sustained by means of pressures on the trainee's body. The latter may be 
implemented by pneumatica I ly 'dr 1 ven elements in the simulator seat (G-Seat) 
and through the gravitational alingment cue provided by tilting the motion 
platform. lAof\on cues may be provided by +he-7ndvement trf^^e^^m. 
motion platform and possibly the actuation of the pneumatically driven 
elements In the G-Sea;t. The indivi dua I research Items have been broken 
out as f o 1 1 ows : 

o Contribution of the individual or/:omblned degrees of freedom 
of motion of the platform to training of the undergraduate 
pi lot trainee. 

■^1 ^a^ 
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o Contribution of the Individual or combined motions of the 
degrees of freedom of motion of the platform to the- 
orofictency measurement of the .trainee, e#g«, periodic or 
final checks, 

o Contribution of the gravity alignment cue to training. This 
cue comes from tilting the platform such that. normal gravjty 
is used as a substitute for the acceleration when the aircraft 
Is accelerating or decelerating, e,g., during landings, 

o Determination of the' degree to which the seat pan*-and belt' 
pneumatically driven elements (G-Seat) may provide the 
motion cues available to the trainee during various training 
tasks and maneuvers, 

o Determination oj the contribution to trailing of the sustained 
pressures provided by the oneumat i ca I ly driven element of 
the studei\t*s seat, * ^ * 

■ \ • ^ 

o Determination 'of the optimum frequency response In pitch, 

roll and yaw of the platform for trai-nlng cost effectiveness, 
i.e,, what^ f idel ity of motion is optimum for training, 

o ^"""^Py^ jmination of^y Ke opt imurn^program for introduction and 
washouT of linear movement of the motion platform and G- 
Seat fpr training effectiveness. 

Mot i on-V I s i on I n te ract i on 

This aVea of research Is concerned with determining the combined ^ 
effects upon training effectiveness of the vision and motion characteristics 
of the device. / • 

Feedback 

ThKs area Is cojicerned with the provision of feedback (knowledge of 
results) to the student about his performance. Usually feedback Is given, 
relevant to some criterion or standard. This area can be divided Into 
categories. as follows: • ^ 

o Freeze Capability - Refers tothe most effective use of the^ 
simulator "freeze" capablll*ty in providing feedback, 

o Sensory Channel - Refers to the determination of the most 
appropriate sensory channel for use in providing feedback 
or knowledge of results to* the student. 

o Instructor Provided - Refers to the determination of what 
Information should be given to the student by the ' I nstructor 
as to the quality of his performance based upon Information 
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he obtains either from his observations of the student's 
activities, panel instruments, contact sc6n# or, In the case 
of the simulator, specific performance measures oS both 
criterion- and diagnostic nature, 

o. Frequency - Refers to the frequency with which feedback should 
be given to the student, i.e., continuous or at discrete 
Intervals during his practice of the task, 

o Time Delay - Refers to the time Interval between the performance 
' of the task and the presentation of Information relative to 
the adequacy orf quality of the performance, l.fe., whether feed^ 
back Is Immediate or delayed, * - 

o Visual Modeling - Refers to the process of exhibiting ;to the. 
student on a visual display hi*s progress through a manetJyer 
as a developing two-dimensional spat i a I pattern on a visua 
* (display. Fop example, while the student ^Is performfng a 
holding pattern using information derived from the Insfrum 
on the aircraft pane] a- CRT display may graph for him his' 
ground path relative to the desired holding pattern. This 
information may be withdrawn as trainee learning progresse 

o Self-Confrontation - Refers to tjie olayback of a trainee's 
performance during a practice trial through activation of 
Instruments and controls of >the 's 1 mu I ator. Playback may be 
either in slow, real or fast time. ' ^ 

<», 

Disorientation Training ^ } • 

This category is concerned with the research Into training to recog- 
nize and cope with disorientation, during flight.' I^t Investigates the 
requirements for a simulator to induce disorientation and the methods and 
techniques for training in recognizing and coping with the problem. 

Auditory Cue, Simulation 

/ 

Tlfis-area of research js concerned with the auditory spectra emanatl 
from the aircraift which may be important in training In several different 
ways as fol lows ; 




o 



As -^an information source^wh Ich' the student uses In directing 
his control movements. 



o' As a means of adding realism tb the training situation. 

o As noise wKich interferes with obtaining Information or 
which may otherwise be detrimental to performance.. 



63 

71 



Kinesthetic Cue (Control Feel) Simulation _ * 

This area of research deals with the Investigation of the decree to 
which the control forces and displacements J n the aircraft being simulated 
should be represented In the simulator. 

Contextual Training 

Thfs area ois concern Is with training taking place within the context 
*of operational tasks and applications. For example^ basic instrument 
maneuvers such as 30"* bank turns could be taught In, the context of an 
overal ririaheuver such as an Instrijment approach. An experimental question 
in this area might then be, shouM: tasks such as 30^ bank turns be 
practiced separately and unlauel^ or within the context of a broader 
ffianeuver? 

Performance Measurement 

Within this category there are five ident i f iabfe, independent sub- 
categories as follows. 

o Check-ride Performance - Extent to which check or criterion 
rides In the sImuHator may be substituted for those in the 
aircraft, k 

o Diagnostic Measures - Measures of specific Items at a 

particular time during a practice session or learning process 
which aVe designed to Identify and isolate specific sources 
Qf difficulty being experienced by the trainee' In »hls* . • > 

acquisition of the criterion skills or knowledge, - 

o Control Output Measures-- Identif icatlo^i of the parameters 
to be sampled and recorded, .e.g., lateral ^st I ck Input, and 
the methods for summarizing or transforming them Into the 
most meaningful ^nd valid fom, e.g., spectral density 
function or 'freqxiency analyses. ^ 

o" Systerr/ Output 'Measures - Identification of those parameters 
* ' fo be sampled and .recorded by maneuver or task, e.g., altitude 
^ or a! rsjieed, and' the methods f pr summarizing them Into the 

most rr^aningful and valid form, e,g. Integrated absolute 
. epror, RMS, ?t^ 

o dbserA/er Records oi Performance - Invest i gat I on'^of the methods 
and tecKnJques^ wherel?y the Instructor pUpt or other observer 
may recgi^d performance thrbugh watching thQ standard Instruments 
' of the- aircraft panel, the contact world 'scene and the actions 
of the trainee* • • " * 

' o Observer Opi'nfon Data - investigation of the methods and 

techniques whereby an observer such as an Instructor pliot. 
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test pflot, design engineer or behavioral scientist may 
record or express his judgments or opinions in a useful 
and rel lable manner. 

o Determination of the relationship between specific and 
detailed diagnostic and criterion measures obtainable In 
the simulator and those* measures ppsslble of being obtained 
by the instructor pilot observing the trainee performance ^ 
In the al rcr^ft. 

Instructional Aids 

This area of research Is concerned with the methods, procedures and 
part-task devices which may be used In connection with a simulator and 
within a training system to bring-about more effective training^ The list 
of. items which may be broken out under this category are as follows: 

o Maneuver Demonstration - Refers to the capability of the device 
for '^playing through" the maneuver In automated mode for the 
purpose of demonstrating Jt to the s/tudent. 

o Instructional Cues - In this Item of research use' Is made of 
the instructional cues derived from task analyses In that 
the instructional cues for each task or maneuver are singled 
out, and made qxp licit for the trainee. Instructional cues 
are defined as the stimuli which provide the Information; 
often in the form of a rule or a set of procedures, which 
enables the learner to perform the behavior described In a 
performance object I ve. It Is the minimal informational stimulus, 
either audible, visual or tactual, which must be supplied to 
the learner in order to enable him to make the desired response. 
The emphasis In this area of research would be the pointing 
out or making explicit to the learner what instructional cues 
are Involved in performing 'the task. This may be carried out 
through ifse of^ the visual and/or auditory display capabilities 
-of the ASUPT or use of other less complex audio and/or video * 
devices. 

o Part-Task Trainers - Research using much less complex devices 
than the ASUPT to train in parts or aspects of a task p^rior 
to ASUPT training. Such devices might include full scale 
panel illustrations or pictures, mock-ups, p<rocedures, trainers 
and audio-visual devices for training In procedural, perceptual 
and cogn i ti ve- type tasks. 

o Prompting and Cueing - Prompts are signals which Indicate that 
the time has come for .a specific action to occur and directs 
the student to perform that particular action. Cueing Is 
similar to prompting but usuaMy refers to a simple signal 
that Indicates time to act. Thus, a cue Is less directive 
than a prompt. 
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Aircraft Dynamics Simulation 

This area of research "dea Is with the degree to which the equations 
describing the motions of the simulated aircraft should be represented 
and implemented in the simulator. It Is di rected essent i a I ly toward 
the dearee to which these eauatlons might be simplified in their 
Implementation in the simulator with possible reduction In required 
computer speed and/or capacity. 

Peer Train ina 

Research Fn which the trainee's oeers may observe and work with him 
while he practices or engages in the solution of problems. This category 
may -be subdivided as follows: 

o Instructional Assist - In which the fellow trainee engages 
' in helping the trainee solve problems or improve his skills 
through discussion, suggestion and critlaue of his 
performance. 

o Dynamic Observer - In which the fellow trainee acts as an 
observer and notes the performance of the trainee 
bein^ observed, the types of errors which develop, and 
can mentally rehearse his own technlaues and approach to 
solving the problem or acaulrlng the skill. 

\ 

Instructor's Role 'and Training 

♦ • 
- — This area of research is concerned with developing informatlpn which 

may be used to make the instructor more effective in his guidance and 
Imanagem^nt. of the student's progress through the training curriculum. 

rt may be subdivided into the" fol lowing categories: 

o Performance Evaluation -.This area Jnc I udes^ research . 1 n. the 

training of the in's^ructor in the evaluation of the performance 
of , the trainee in fhe simulator and in *fhe aircraft and the 
[j . re lationshfip between the performance ^measure's obtained In each 

of these training situations. A greater array of performance 
evaluation means is available to the . I nstructor In the 
simulator than in the aircraft. Instructor training In the 
use of diagnostic and 'criterion prof Iciency^measurements 
in the simulator is the concern of this research area. 0*f 
particular concern is the 1 nte^rpretatl on and translation of 
the more detaiTe~d measures taken In the simulator into ' 
instructional guidance and" performance ^va I uat ion in the 
ai rcraft. 

o Use of the diagnostic Information provided 'By the simulator 
performance measures for either extemporaneous or 
— ~ -^standardized instructions and^ guidance to the trainee. 
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o Degree pf Instructor Participation - This research Item 
refers to the degree to which the Instructor actively 
participates in the ongoing practice of the trainee. On 
the one hand the instructor may demonstrate the task, 
talk the student through maneuvers and provide evaluative 
and directive Information throughout the course of the 
student's practice. On the other, he may adopt a passive 
role in which he provides information only when questioned 
by the trainee and injects himself Into the straining 
practice only to assure safety. 

o Instructor Motivation - Investigation of the incentives and 
awards which may be used to increase instructor Interest and 
enthusiasm* 

o Instructor Standardization - Research concerned with training 
and measuring the instructor's ability such that he maintains 
his effectiveness at a defined and established level. 

o Degree of Instructor Task Automation - This Item ^refers to 
the analysis of the instructor's task and the al Ideation of 
certain of these to automatic execution by the training 
simulator. The types of functions and tasks that fall 
within this category are briefings, demonstrations, 
performance evaluation and assignment of train Ing- tasks. 

o Instructor els Training Manager - Refers to the Instructor's 

control of the progression of the student through the syllabus i* 
based upon student performance relative to specified criteria | 
using specific task descriotlons and criteria and having i 
available to him detailed knowledge of the student's progress^ 
through diagnostic as well as criterion performance measurement. 

o Instructor Station and Role In UPT Simulation - Refers to such , 
questions as whether the instructor should be located In or out 
of the cockpit and the optimum design of the instructor's 
console. 

Relationship of Trainee Traits and Methods of Instruction 

This area of research is concerned with the investigation of 
relationships between methods of presenting the training, materials and 
certain measurable traits of the individual. For Example, measurable 
traits such as visual field dependency, manifest anxiety or perceptual 
and cognitive style may be hypothesized to be related to the appropriate 
method or mode of instructlort to be used for a given trainee. ^ 

Cognitive Pre-Tralning 

This area refers to extensive briefing or complete understanding 
of the perceptions and actions required for completing a task before 
It Is attempted In the simulator or the aircraft. 
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Sequencing of Training Tasks^ Maneuvers and Phases 

This area of research Is concerned with the ordering of the tasks, 
maneuvers and phases such that optimum transfer from fundamental skills 
and knowledges to tasks of greater difficulty Is obtained and results 
in the rnost efficient progression through the syllabus. Specific research 
questions may range from whether instrument flight training should 
precede or follow contact flight training to Investigations which seek 
fs> determine the hierarchy of skills In the optimum order and methods 
of teaching these basic skills. 

Trainee Motivation ' . • 

This area of research Involves investigation of the, role which In- 
centives may play In bri nging'about more effective tra-lning. These may 
involve competlon among trainees or other Incentives and awards for 
accomplishment. 

Extension of the Training Syllabus 

This area is concerned* with determining the emphasis ,to be phaced 
upon training tasks and maneuvers which are avoided in the aircraft for 
reasons of safety, .and those which require control I Ing' the aircraft to 
the limits of its performance and structural integrity, e.g., control * 
at the €r it lea I airspeed limits. 



Adaptive Tral>nl 



^ ^' In the simulated environment, UPT may be adapted In various ways. * 
'Under either automated or manual control, the actual task, -the syljabus^' 
and/or simulator characteristics can be modified. The specific Items 
In this area are: ^ * * 

o Individualized Instruction - Determination and description 'of 
the specific tasks to be taken up In the syllabus, specification 
of criterion performance either^ normative or administrative, 
^ and progression of the student through the. syllabus on a 
performance^ to criterion basis with provisions Ifor branchlnc^ 
.back for re\^rsal of tasks. ^ ^ , ' 

o Machine Adaptive Training - Adjustment of the icharacterl sties 
- . of the training device or media or task such that the trainee . 
progresses to criterion performance at his owo Individual" pace 
based upon measurement of his progress. Adjustment of the 
training situation Is customari ly such that the tralnee^s task 
Is easy during his initial practice trials and Is adjusted to 
become difficult based upon his progress 'unt 1 1 he reaches 
criterion performance. ^ ' 
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•» ' APBENDIX C • . . . 

' \ • 11 fe. Sclenpes, Inc. 

' ^ : • Cptftract F4\609-73-.C-od38 

Asslgnrnent^of 'Priorities TO, .V* , ' > 
/Train! hg Prbblems » • • 



The attached 1 1st {jescrfbes areas qf ^research wbt'ch^.have b^ert J dent I- 
fled as those which. are pressing. In terms df' their /fmportance for making' 
pilot training more' effect! ve, ' Resear-ch^ results providing definitive Infor- 
mation }n these areas would be h.ighly usef ul' In tnaking decisions as "to the 
way training should be condijcte^ ahd the .type o/ 'j"raf(>ing equipment; that* 
should be used, . " ^ • ' .■ V • • .-^ * ' * - , 

What is needed now j-s an qrderiog of tb0 listed. Items In terms of 
their importance t^' increasing the j^f.fectrvenes.^ of be'grrinJng qr under- 
graduate pi lot train'lng, that particular: ti^em v/h}ch the ra;l'er feels should 
be taken up first in an .undergraduate pX'^t '''^^IA^^Q res^2ir^?:.H program should 
appear at the -top of the list, ;The^'feasls for: assigning pr^^oritlefs should 
be the Importance tff the re.searctv'l tem for Jts •poten^ffaj /for 'increasing 
training effectiveness! , Increasing tratnlng effectiveness \is d6f}'ni|ed.a 
bringing the s-tudent to cri'terlon prAficjency Jn .^ess tralnfng'tl^ a+ ^©s 
training costj ' • ^. ^ „ . \ . ' ' 



s 
ess 



The question pos^d to the rater Is - what Is. the particular ord^r in ^ 
which the Items of research, .shbai d be^accqmpl Ished In order to provide ' 
the most useful i^nformatlpn t6 admin f-strators In maWlng training fJrogram..- 
decisions. It' is hoped fhat a reasonably concise and *unamblgloUs definjt'lon 
of each of the research* areas hatd'been ^Iven So that rfhey inay. b& reviewed * 
and priorities assighed, . • , / - - \ ^ \ 

Each of the. items, has been listed |eparaffe ly or) a 3.^-5 tarcf so tf^a+ , 
they may be sorted- and arr^noed In ordpp of priority. ^As'you receive 
them the card?.wi ll be- in'the same ord^r as the itjBfhs in the atfached J'lst^' 
The procedure fpr sorting, is to select a given card and'plac^ It'face up 
before you. Each of the other ltdms (cards) Ms exarrtlned and/a Judgment * ^ " 
made as_to whether^ it Is of g reafer or ' lesser _pjriorl,ty tharf the f irst 
chosen (reference) card.; Any IterrTwFiTch is judged to be of higher priority 
than the* reference card is placed to the rl^ht of that card while those . 
Judged tq be of I.esse? p.rlt)rl ty are placed.*tc^ the left. At the 'conc I usjon 
of this sort the Vlgtltx hand staqk of cards Is taken up and sorted In jire- 
.clsely the same manner. That J;S, a. reference card Is chosen and placed 
face up' before the rarter. The remaming cards Jn the stack are then sonted 
into two piles, one on either side of the reference card, designating either 
higher priori ty. or lower ..priori ty than ,fhe reference card. This, procedure 
Is repeated fqr all stacks untj I no one has^more than three or four Item 
cards in it. These three or four may then be ranked from high to low in 
priority.. The total stack of oards 'is then reassembled \from greatest to 
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least priority. \f M is convenient the items "shoul d be typed into a. list. 
In the priority order shown by the cards. If t()fs Js hM 'ccnvenjen^t, .the 
prioritised card stack should be.secured f i rmly *ahd:retu^ried^tQ Life 
Sciences in this form. It i^s preferred, both for" coftvenience' .In tuailj.ng 
and' for .insuring that the iterms will not be gotten out. of cfj^'der,, thM "thev 
t-yped list be made. 
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Ranking of- Trailing Problems Based 
Upon Median Rank (40 Raters)' 



ITEM 



... 2. 
•: 3. 

■ 5; 
.6. 



— f 

i 



8. 

I'Ov 

ru 

12. 

,13. 

14. 

-15. 

16.' 
17. 



18. 

J.9. 
20. 

.22. 



23. 
24. 

26* 
27.: 



Content of tK6 Visual Dl splay 

.Motitfn-Vision Ihteraction 

. Oual Ity of the Visual Qi splay • 
Performance bteasurement-S^stem Output 
Measures ' . " ' 

Sequencing' of Training, TaskS 
Contriibljtibrt -q^f the Individual .or •■ ^ 
"combined Degrees of Freedom of Plat- 
form Motion to Training' 
Instructor Training - Performance 

. Eval uation . ' ' 

'^Cognitive' Pre-Tra in ing 
Performance Measurjement Diagnostic 
Measures 

' Adaptive Trei in ing - Individualized * 
■ instruct i'on - ' . , v • 

Feedback - Self-Confrontation < ^ ^ 
.Feedba'ck - Visual *Modeiirfg 
"Instructor -Training - Use_„of Diagnos- 
tic Information for Student Guidance 
, Feedback - Sensory Channel 
'Instruct ionaj Aids Instructional 

Cues 

^ Contextual Traini,hg 

..Corvtrib'ution of the Individual or ^ 

XomblnecJ Degrees of Fr^Bdom of Platform . 

' Motion ^,to Proficiency ^Measurement 
Instructor Train,ing - Degree of 
rnstri^ctor Ra'rt I cr pat fon ' - ' 

Feedbrack - Instructor Provided 
Fiefd of. Vfew * ^ " ' • • / 

•/Adaptive Training - Machine 
Determination of the Opt ImOm Ff d^J i ty of 
Response in Pitch, Roll and *Yaw of , 
the -Mot ion Platform - ^ , ^ 
jnstructor f>>1oti vat Ion 
Performance ^teasur^ment - Relationship 
Determination of the DegVee to Which 

•G-Seat May Provl^de Mbtion Cues 
Feedback- Frequency 
PerJormance Mfeasurement «- Observer 
Records of Performance * 



MEDIAN 

6.0 
I 1.0 



2 ^ 

10.75 
I 1.50 



1 o c 

12.5 


19. 75 


12.5. 


12.75 


14.0 


1 1 .00 


14.0 


1 1.75 


1,4.5 


' 1 1.00 


15.0 


9.75 


18.0 


10.00 


18.0 ~ 


7.50 


20.0 


14.25 


2Q.0 


• 8.00 


21.0 


12.00 


21.5 


18.75 


21.5 " 


8.50 


22.-0 


9.p0 

1, 


.24.0 ^ 


17.50 


24.5 


18.00 


24.5' 


18.00 


'25.0 


16.75 


25.5 


I0;75 


25.^ 


- 16.00 


26.0 


23.75 


27.5 


9.50 


27.5 • 


7.75 


28.5 


12.75 


28.5 


13.25 
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Ranking of 'Training Prpb+aqjs Based 
Upon Medlars Rank (Cont*d) 

iTEM MEDIAN Q 

'28, Determination of the Opt I mum- Program for ^ 
^ Linear Movement of the Motion Platform 29.0. 13.75 

29. Determination of the Importance of the 
Auditory Spectra as an lnformatl6n 

Source 29.0 16.00 

30. Instructor Role as- Training Manager 29.5 16,25 
31/ Peer Training • Instructional Assist 29.5 11.75 

32. Performance Measurement - C^trol 

Output Measures ' ^ 30.0 14.25 

33. Instructional Aids - Part-Task 

Trainers 31.0 '14,0 

34. Depth of Field of View In the 

Visual Display ^ 31.5 ^ 18.50 

35. Instructional- Aids - Prompting and 

Cueing * , ' ^ 32.0 ^ 12.25 

36. Instructor Standardization ^ 32.0 16.25 

37. Performance ^Measurement - Checkride . 

performance '32.? 10.00 

38. Determination of the, Contrl but Ions to 
, ' TVfaining of the Sustained Pressures . 

Provided by the G-Seat , 33.0 17.50 

39. Degree to Which tfte Control Forces 

and Displacements Should be Represented , ^ 

In the Simulator 33.5 11.50 

40. Relationship of Trainee Traits and 

Methods of Instruction ^ 33.5 20.75 

• ^ 41. Instructor Training - Degree of 

Instructor Task A'utomatlon 34.0 19.50 

42. Peer Training - Dynamic Observer ' 34.0 - 13.25 

43. Instructional Aids - Maneuver 

Demonstration 35.0 15,50 

44. Extension of the Training Syllabus 35.0 14.75 

45. Contribution of th'e Gravlt'y Alignment ^ 

Cue to Training' - ^ 35.0 16.25 

46. Disorientation Training ^ ' 35.5 11.25 

47. Feedback - Time Delay 36.0 10.00 

48. Trainee Motivation . • ' . 36.5 20.50 

49. Performance Measurement - Observe^ ' • 

Opinion Data • 38.0 , 8;00 

50. Determination of the Importance of the ( 

Auditory Spectra as a Means of Adding \ , 

Real Ism to the Training Situation^ 40.0 "5.50 

51. Determination of the Importance of the 

Audlto'ry Spectra as Interference or Noise 44.0 10.75 

52. Aircraft. Dynamics Simulation 45.0 18.75 
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MEDIAN • 


Q 


47.0 


7.00 


47.5 


10.25 


48.0 


7.25 







Ranking of Training Problems Based 
Upon ^4edlan Rank (Cont^d) 

ITEM 

53, Feedback-Simulator Freeze Capability 

54, Color In the Visual Display 

55, Instructor Station - Location and 
Design 



Note \. — ^ 

In this table the median refers to the mld-polnt In the dlstrlbjutlon 
of ratings giveq by the judges. It Is that ratfrtg value. at which one-half 
or 50^ of the judges ratings of the particular Item were greater than the 
value and one-half of them were lower. The Q'^value In the table Is a y 
measure of the variability of the judges' ratings and is called the semi- ^ 
interquartile ranger* The formula for cue-ls^Q = Q3 - Q|« 0| Is the point / 
one-quarter through the distribution of ratings, I.e., the point at which 
15% of the ratings are above thl^s value and 25% are below. * Q3 Is tijie point 
three-quarters of the way through the dlstr>lbutlon. Thus, the difference ^ 
between Q3 and Q| divided by two Is a measure of the spread or variability - 
of the judges* ratings. / 

. / 

• ■ / 
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APPENDIX E y 
Scenario for Flights of Instrumented T-37 Aircraft 

y , Sortie No; I ^ * ^ 

Record runway barometric pressure and temperature, dat^, time. 
Take-off. 

Turn on recorder, ' - 

Gear-up - record, ( I ) ^vent no./ (2) fuel quantity', (3) tiJrbulence 
level, (4) "climb". 

Climb to 15,000 feet at a constant 180 KIAS, Perform 4-2 minute 
trials during climb. TrFm before each trial as necessary. 

Fly to area where maneuvers c2Mi..i)e flown. 

Update event mark - record, (I) ^event no,, (2) fuel quantity, (3) 
turbulence level, (4) ^'descent". 

Descend at a. constant airspeed of 2p0 KIAS to, 2,000 feet. Penftorm 
4' - 2 minute trials during descent. Trim before each trial as 
necessary. ' ^\ * ^ 

Trfm to 200 KIAS^ straight aiid^level, 2,000 feet. 

Record (I) event no., (2) fuel quantity, (3) turbulence level, 
(-4') trial order A. , ' • ^ ^ - 

Fly 2 cninute trials in order A, Trim before each trial jas ^ 
necessary. ^ ^ / 

Recor(i6 ( I ) event nq,, (2) fuel quantity, (3) turbulence le>^'el, . 
(4) "straight and level to base", 

■* ' . ✓ 

Return to base at 2,000 fe^t. Perform 2 mlnuta trials straight 
and level during return. Trl^m before each trl^Jj^as necessary. 
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, ■ . Sortie No. 2 

/ 

\ 

L Record runway, barometric pressure and temperature,^ date, time, 

2. Takeroff, 

3. Turn on recordj^^ 

4. Gear-up - reco^^( I) event no., (2) fuel Quantity, (3) 
turbulence level, (4) "climb". 

' • ' ) 

5. Cirmb to 15,000 feet at a constant 180 KIAS. Perform 4-2 
minute^ials during climb. Trim before each trial as necessary. 

6. Fly ^to, area where maneuve-rs can be flown. 

7. Record, (I) event no., (2) fuel quantity, (3) turbulence level, 
(4) "descent". 



'8. Descend at a constant airspeed of 200 KjAS to XUOO feet. 

Perform 4-2 minute trials during, descent. Trim\efore each ^ 
trial as necessary. ^ ' . J 

9. Trim to 200 KIAS. straight and'leveL, 2,000 feet above groundT 

10. Record (I) event no.-, (2) fuel quarttlty, ( 3). turbu lence level, 
- (4) trial order B. ■ . ' 

11. Fly 2 minute trials In order B. Trim before each trial as 

necessary. ■ ., 

/• 

12. Record (I) event no., (2) fuel quantity, (3) turbul^rtce level-, 
( 4 ) " st ra i ght an d~ I'ewl ' to base". • ° " , 

13 Return tcJ base '-at 2^^000 feet. Perform 3-2 mlnufr\rlals straight 
and level during return. Trim before each trial as necessary.' 



/ 
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Sorties 3 i 4 



Repeat Sorties No. I -and 2 except perform straight and level and 30 
banirtums at roo KIAS. . * • ' 

Sorties 5 i 6 

E^'ljeat Sorties Noi^l and 2 except perform straight and level and 30 
bar^'^. turns under contact conditions. ^ 

N . ' Sorties 7 i 8 



Repeat Sorties No. I abd 2 except perform straight and level and 30 
bank turns_*are at 100 KIAS and under contact conditions. 



.6 •» 
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APPENDIX F 

Eye-movement Recordings and Data Analysis 

It is recontnended that the NAC Eye Mark Recorder* Ls to be used 2is ^ 
method for gathering data for the Identification of the objects comprising 
the content of the visual display which can be used as referents by the 
pilot In control of his aircraft. The NAC Eye Mark Recorder provides^ 
record of a primary field of view 60^ ^n azimuth of the scene directly 
forward of the pilot's head position and an Indication on this field of 
view of the pilot's fixation Doint within the scene. Specifically, this 
recorder is an optical device which focuses an Illuminated reticle re- 
flected by the movement of the eyeball so that it always coincides with the 
visual I Ine-of-regard. This is superimposed on the primary .Image and may 
be recorded on 16mm film or video tape. The video tape recording method 
is recommended because of Its capability for Instant replay. 

The general approach recommended for the collection of these data Is 
to require ijistructor pilots to "fly representative undergraduate pilot 
training maneuvers while their eye movements are recorded. The specific 
maneuvers recommended to be flown are (a) straight and level,. (b) steep 
turn, (c) complete (deep) stall, (d) lazy eights, (e) a shaMow turn, (f) 
pattern, approach and landing, and (g) formation flight. 

Instructor pilot subjects will be selected who are current In the 
maneuvers to be accomplished during data collection. These pilots wN I be 
briefed on the general .characteristics of the eye mark recording equipment 
and such safety procedures as are required. This briefing will Include 
Information, as to the purpose of ihe maneuver and data collection procedure. 
The pilots wl I f be Informed as to why they are beinq monitored and that 'at 
the conclusion of a flight they will be asked to review the recordings of 
their eye movements and to relate what elements of the visual scene they 
were using as they piloted their aircraft. ^; 

An important phase of this investigation Is the post flight review 
by the pilot of his eye-movement recordings. It Is necessary, on a moment- 
by-moment bas5s,^for the subject pilot to identify the particular element 
In the contact world scehe which he Is using to provide him with Infor- 
mation for control of specific dimensions of movement of his aircraft. It 
Is also necessary that he Identify any re I aid on ships either static or 
dynamic between the external world referents and those flx6d to hts air- 
craft. These -referent points will serve to Identify objects, points, lines, 
fegjjjres or characteristics which may then be generated In ♦the contact 
j|^^^&rjd /scene provided by the simulator.. These Identified referents 
^^^Sr^wviae the basis for investigations as to the content objects and 
their location in subsequent research Investigations. 

! It Is also hypothesized that It Is a requirement of the I lot trainee 
t6 learn to Identify and extract the pertinent referents from visual 
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"noise" In* the contact world s^cene. This nol.se is comprised of other 
objects or conditions which serve to obscure the referent and interfere 
with fine discri/ninations of its relationship to other referents. During 
the po§t flight interview the pilot must be asked {whether he can identify 
such .condition in the videotape he reviews. They may be such factors as 
haze, atmospheric attenuation, low c6ntrast, or any other which tended 
to interfere with or degrade his visual pferceptions. 

Of particular interest in the post-flight interview is information 
relevant to the peripheral cues observed and used by the pilots during the 
flight. The long standing belief in the importance of peripheral cues 
(Hasbr'ook i Young, 1968) has been recently reinforced by *data collected 
by Life Sciences, Inc. in which pilots eye-movement records were taken 
while hovering the helicopter. These records show that a pilot could 
fixate steadily on a given point on the ground plane for a long period of 
time whFf^ controlling all six dimensions of movement of his aircraft In- 
dicating great reliance on peripheral visual Information. 

Although the NAC Eye Mark Recorder Is equipped with a 60° field of 
view lens the peripheral vision of the human operator extends beyond that 
field and it is quite probable that Information Is available and utilized 
by the pilot which rs not recor*ded. The post-flight review of the record* 
Ings and the interviews should cover this possibility, i.e., the pilot 
should attempt to recall and elaborate upon any peripheral cues which are 
not visible in recordings at any point he feels neces^sary,* 

Although the I ine-of-regard of the pilot can be established with the 
Eye Mark Recorder this does not establish the fact of the object or area 
on which the eye Is focused. That Is to say, that the I Ine-of-regard 
reticle of the recorder resting upon an object does not necessarily mean 
that the eye is' focused upon that object, although It Is highly probable 
that It Is. The pilot, therefoij-e, must be able to explain precisely what 
object or point he was focused upon during the portion of the maneuver 
which is visible on the video tape. The pilbt may provide narrati ve- Infor- 
mation on a, voice tract of the video tape during data collection. This 
narrative fs not deemed critcally necessary but could prove helpful. 
If there is any Indication tha"£ such a narration Is Interferring with 
the pilots normal flying of the maneuver it should be discontinued. ' 

Although It Is preferable that a given maneuver be accomplished la 
"discrete" fashion with Immediate return to the flight line for playback 
and interview, the data collection sequence Is open to variations because 
of t^he pract lea I i ties,' of flight data collection. For example, data nilght 
be collected, on taxi and •take-off , climbout, and Immediate approach and 
landing with return to the flight line and playback and Interview covering 
this sequence. If Is recommended, however, that to the maximum extent .,-1^ 
possible, an extended sequence of maneuvers not be accomplished since It 
would require a greater amount of recall by the pilot and certain factors 
may be forgotten before the Interview ses^l^on. 




■ Documentation of eye point data in the total field of view will be of 
direct assistance in establishina the visual environment of the pilot and 
can be utilized in other programs. For example, the data could 6e used 
by instructor pilots for directing the student's attention to particular 
areas of the visual field during training. However, first and primary use 
of the records and the interview data is in generating hypotheses as to 
the visual image content and fields of*vfew to be tested In ASUPT^ 

The. outline of the procedure for carrying out the eye-movement data 
collection and analysis is as follows: 

^^*o Purchase NAC Eye Mark Recorder and modify for helmet mounting 

*o Preliminary try-out of equipment and procedures with HRL/FT 
personnel 

o Selection of instructor pilot subjects . 

o General or ientati on* and safety briefing 

o Selection and data collection on maneuver 

o Playback and interview concerning maneuver 

o Analysis to determine field of view 

o Formulation of hypotheses for test 



*Accomp I i shed as of this date 

I ' At the time of the writing of this report the eye-movement recording 
jbrogram Is going ahead at HRL/FT. As Indicated In the list of procedures 
To be accomplished the NAC Eye Mark Recorder was obtained and modified 
for helmet mount. Prfelimlnary data collection fb try out both the Eye 
Mark Recorder and the video tape equipment has been accomplished using 
ground vehicles. The means for mounting and operating' the eye mark 
equipment and the recorder in the TF37 aircraft have been specified. 
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APPENDIX G 



Functional Description of^AVIT Device 

A schematic drawing of the existing AVIT is given In Figure L 

The stimulus devices to which student responds are: ^ ^ 

!• An J8 X II Inch rear-projection screen on which photos of the 
basic flight Instrument panel of the 1-37, or other qhotographl c, written, 
or diagrammatic material, as appropriate, may be presented; 

2. A head-set for audio messages; 

\ 

5. A set of six "response evaluation" lights In two rows of three, 
the top row labelled ROLL OK, PITCH OK, POWER OK against a green back- 
gcound, and the bottom row labelled ROLL ERROR, PIJCH ERROR, POWER ERROR 
aqai ns±^ re^ background. — — .~ 

The response mechanisms that the student may use ard: f 

\. A sTmDTated stick and throttle, with a ^'trigger" sWltch on the 
stick. 

2* A row of 5 response buttons, labelled A, B, C, 0, and.R. 

' In addition to the above the student -has available a SOUND REPEAT 
button, which allows him to repeat an audio message, and an EXPOSURE 
CONTROL d'lal, the use of which will be described later. 

Trainer Mode — 

The AVIT operates in a number of different modesT^whl ch are selected 
automatically by the program material; The basic TRAINER mode will be 
described first. — 

^ Early in an AVIT program a typical programjsegment (Figure 2) ml^ht 
be as follows: on frame 20 of the program, following some introductory., 
material, the student might see^n the screen a photo of the instrument 
panel with all the InsWuments ^showing the readings that are correct for. 
a particular flight cond>+lon, as shown Irl Figure 3. The accompanying • 
audio message (A-IO) reviews the readings and Instructs tha student to , 
press his trigger switch to advance the first prol^fem fr^me, frame 21. 

On frame 21, al I instrument readings might Jbe n^hnaf except for a 
small increase in altitude. There will be no sound message with frame 
2^; but the student w*i I I have been Instructed previously that he^ Is to 
make;the appropriate control movement's to return to his required fH^t 
condition. He moves the controls to the position he desires and presses 
hls/trlgger switch to "register" his response, o 
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If he has responded Incorrectly the AVIT,*wll[ move automatically to 
frame 22. Thevisual material Of\ frame 22 I I , be' the -same as 'on frarfve 21, 
but there will be an audio message (A-l I) which will call attention to the 
specific Instrument readings req-u I ring corrective action.. In addition, the 
appropriate Vesppnse evaluation' 1 1 ghts will come on. The audio message 
might be: ' ' ' , 

"Here your attitude, airspeed and heading are 'all right, but you 
are AOO feet above your assigned altitude. ' Check the I l^hts^ to 
see where your specific error was and then press the trl-gger to 
. return to try the problem again''. ' * 

When the student presses the trigger the AVIT returns to frame 21, 
and the response evaluafTon lights go- out, o^ course, 

• . ^ 

'When the student has madq the right response the AVIT^wlll move to 
frame'23. The visual presentation on fcame 23 will* be the normal set of 
readings as on frame 20, and the sound message (A-12) will specIMcalJy * 
confirm the correctness of the response. Ror example, A-12. might be: 

r 

"Very good. The only problem was a 100 foot error on the altimeter. 
Forwar^d .stick was needed to begin a return to 15,000 feet. For thife small 
decrease In altitude, no power change would be necessary. ,Very well, press 
the t^-lgger switch for the next problem." 

When the student pressed the trigger, the AVIT would move forward to 
•frame 23, and so on. 

Later In the program, the conf ^^rmatory frame would be orrtltted, and "the 
student making the correct response would move directly to the next problem 
frame. The green lights would come on briefly to confirm the correctness of 
his response, and he would have previously been instructed that tie should 
respond to the new problem directly. Such a program segment Is. shown in' 
Figure 4; ^ . * ' , 

The program segment structures Just discflssed are of the simplest kind 
and the AVIT Is capable of handling much more complex ^structures. Figure 5 
shows a 5-question j'pretest" structure, arrarvged so that the student who 
passes the pretest with a. perfect score will automatically skip a particular 
Instructional sequence. Frame 40 Is the Introductory frame for the pretest. 
Frames 41, 42 a^nd 43 are a "warm up" probleni on which. an error will be 
"forgiven", and the audio message on fram^3 (A-27) would warn the student 
that from now on It. was "for real" and ask him to press his trigger to 
begin the test. - ^ ^ 

Jn Figure 5, the problem frames as sucb are frames 44, 46, 48 and 50 
and 52. Any error leads the student to frame* 55, where the instruction on 
the particular topic begins. The student'who gets all 5 problems right 
coTnes to frame 54. The possible excursion from frame 54 to 55 represents 
the fact that we might of fer the student who reaches frame 54 the option 
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of goinq~¥h"rduqTi the instr ucFion on the particular maneuver if he desires 
7p""do so, even though he hai passed thte"pretest, . (How the studerTt Indicates 
his choice on frame 54, will be discussed under DIDACTIC mode later.) 

Both^ students and instructor-pilots have been enthusiastic about the • 
program structure shown in Figure 6, usedjn connection with the EXPOSURE 
CONTROL mentioned previously. Basically, the structure is a 1 problem 
•test seque?rce w^ere any error returns the student to the start of the 
sequence. On the introductory frame (80), however, the student's attention 
is called to the EXPOSURE CONTROL switch, w^Uch has the following settings: 
SELF-PACED, II sees, 8 sees, 5 sees, 3.5 sees, and 2.5 sees. The dis- 
cussion so far has assumed that this control was set on S^LF-paced, and 
that, therefore, the probl,em remained on the screen until the student 
pressed the trigger switch to register his response. On the timed settings 
the trigger switch Is automatically pressed (in effect) at the end of the 
time set. If the student has not moved the controls to the correct 
position at the expiration of the set time, he will score an error, of 
course, ff he is.cprrect, he mov^^s on to the next problem. (He can, of 
course, register his answer In the normal way, by pressing his trigger . 
switch, even with the EXPOSURE CONTROL timer on, if he does so before the 
time expl res. ) 

a 

The EXPOSURE CONTROL timer worKs only on problem frames (such as 81, 
83, 93 in Figure 6.) It would; not work on the "vrong answer" frame such 
as 82, so the student who is wo.rking with a low. setting, sych as 3.5 sees, 
gets a "breather'' if he makes an error, at least. 

The student who gets all 7 problems correct emerges at frame 95, 
where" he 4^ congratulated by the audio (A-68) and offered the opportunity 
to begin anbther simillar sequence, perhaps at a lower time setting.' In 
one AVIT program there are three such seven-question sequences In a row, 
and- if the student w^nts more practice after completing all three,' he can 
elect to go back to "fhe first set of exercises, and so on. 

Some appreciation o? the flexibility of the audlo^system used in the 
AVIT may be obtained from the last two patterns discussed, particularly 
that shown in Figure 6. Visual stimulus presentation devices that permit 
flexible presentation have been available for some time, but audio and 
audio-visual devices have generally been locked Into a .f Ixed.. sequence. In 
contrast consider how the audio apparatus in the AV'IT must function to 
permit the pattern fehown in Figure 6. Wher) frame 81 appears on the screen, 
the audio device (a; tape player) has the tape so positioned as to be ready 
to play audio message A-61. If the student gives the right answer, *the 
AVIT logic switches the tape p layer*, to FAST FORWARD and runs to the be- 
ginning of audio message A-62, but does not phay it. I f an error Is made , 
(on frame 81) the t^pe player* si-mply switches to PLAY and plays A-61, 
sifopping automatically at the end of the message* When the student, on 
frame *82, presses the trigger switch to retj^jrn "fo frame 82, the tape 
player must ^|<tch to REWIND, rewind message, A-61 arid be ready to play It 
agai.n.^K t^te'student makes qn error on f*teme 95 and so goes to fr^me 
94,^ the»tape recorder wiJI play audio message ♦A-67, of course. When the 
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student presses the trigger to return to fname 81, however, the- tape — 
player must now rewFnd 6 messages, and stop in a position reedy +o play 
A-'GI again, ' " 

DIDACTIC MODE 



Mention was made of the five lettered buttons B, C, D, and R» 
The AVIT is a fuf^ther development or modification of the AutoTutor Mark 4, 
which is a branching teaching machine. The student using a Mark 4 
responds to multiple choice questions by pressing buttons and the Mark 4 
"branches" the student to appropriate visual material on the basis oi the 
student's response. A similar capability is. built into the AVIT, and 
allows it to operate in the DIDACTIC mode. The DIDACTIC mode has two major 
purposes:, 

I, First, for ^instructional seauerices where a multiple choice question 
is appropriate. For example, the visual presentation might be the altimeter 
alone, and the student would be required to select the altitude shown from 
4 choices appearing as numbers on the ^creen, with a button designation 
beside each. The si-udent choosinq the right answe/r would advance to the 
next Instruction frame and question. For the student who makes an error, 
two options are available. In the DIDACTIC mode, the AVITcan operate In^r 
a full branching sub-mode, so that each button leads to a different fram^ 
or in a "right-wrong" branching mode in which the right answer leads to 
one particular frame, all wrong answers to a single second fr*ame. The 
choice depends on whether a single correctional frame Is appropriate for 
all the errors, or whether It i$ desired to supply specific correctional 
.material for each error, QThe reader will recognize that the seq^uences , 
shown for the TRAINER mode all use "right-wrong" branching. In fact, . 
the fuJI branching option is available in the TRAINER mode also,^but we 
found no practical use for it in the TRAINER mode and would not propose to 
Include t.lvPs option in an AVIT designed for heUcopter Instrument training,] 
In any event, the inclusion of the DIDACTIC mode* of operation allows 
"conventional" branching programmed instruction to be freely Intermixed., 
with instruction in the TRAINER mode,' _ - _ _ 

.2. The second use made of the Dl DACTIC. mode Is where the student. Is 
offered options as"to how he wishes to proceed through the program. Such 
choice points were shown in the orogram segments shown, in Figures 5 andL 
6, where the student may elect frame 54 to go through an Instructional 
sequence he is "entitled" to skip by_ havi hg ^passed the pre-test for the 
maneuver, and* in the segment shown In Figure 6, frame 95, where the, 
student may elect to take another test sequence or to 50 on to the next _ 
maneuver. ^ . — 
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^' OTHER MOailES 



Str^etly speaking, the AVIT has several other modes of operation which 
will be mentioned briefly. They and their uses are largely se I f-e)^planatory. 

MANDATORY FWD mode (MF) no option, used for procedural frames* 

Excursion is initiated by pressing the trigger switch. Frames 
40, 43, 47, 49, 51 and 53 in Figure 5 are examp^les. 

'MANDATORY RVS mode (MR) No option, used for procedura L frames. 

Excursion is Initlgited by pressing the trigger switch or 
the R button. Frames 82, 84, 86, 88, 90, 92 and 94 In 
FIcJure 6 are examples. ' ^ ' 

AUTO ADVANCE mode (AA) In' this mode, the AVIT advances automatically 
to the next frame (in the logical, not necessarily the numerical 
sequence) as soon as the sound message for the frame is 
finished. This mode has two usesl First, it allows an AVIT- 
program to contain segments of cohvervt Joha I audio-visual 
nate^^ial similar to that 'used >Ith synchronized sound-slide 
devices, and this frequently a useful programming resource. 
The second use of the AUTOMATIC V\DVANCE erode is fpr excursions; 
* ' that are. too- loijg to be covered in a single jump. The' longest, 
sing le excursion -possible in the AVIT for th? visual material 
frames forward or backward, and 31 messages forward or 
29 backward^ on'the audio. If a longer excursion Is required, 
the programmer sijnp I y includes an AUTOMATIC jA^DVANCE .frame at 
the appropriate point, and ^automat leal ly begkQ^^new 
excursion from thaf,poI^nt. With thi s .resource excursions ^ 
_ . of ^ny reasonable length, become possJb le. 

It should be re-emphas.ized that ttfe sfe lection of the desired mode on 
a particular frame is entirely automatic) requiring no attention from the 
s.tudent or f/offPa supervising Instructor. The"only control the student or ^' 
instructor has occasion, to use on the AVIT^Cexcept for the response devices, 
of course) is the tXPOfuRE CONTROL -switch. , ■ . 
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APPF-INOIX H- 

A '^oconfTondad '>sion. for 
rxnerinontal Ctudtes Us inn ASIJPT 



t Brief, Poscri^^d heroin is n doslon which lends itseLf to studies of 

the raTaTTonshins of oi loti'no oorforn.^nca fo conditions of simulation nrovidod 

-trcr^SUfn. — T-t re \ T\ io rxr\? i '^rx-^^V^^^-h^-'^M^ which v^orl^3_ 

are described, for an illustrative stTjdv. ADniicatTons and exH^fiblur.^ uf Hi.. 

' TOdul? thoutih .thexr-TQf^va-i- n to tc Goocifi o d, ^ -Po-^touch Pd on hripflv . 



General. In the fraininn of ol lo^s, aj; w^ll as in oerfornanco sfydios 
us inn exoerienced oilots, the conditions under which pi lotinn, tasks are ^ 
oerformed in ASUPT will often be of an -i ncronenta I nature. In perfornance 
* studies' involvina hiqhiv proficient" pilots, however, oroq^es^*'' de^lradations 
or reductions in features of sinuldtion mav be introduced f^or the purpose of 
deterrninino the extent to which nerforranco ^f pi lotinn 'tasks den^nds on such 
conditions. In either of these cases, the desinn described in thrs paper 
would apply. 

Ch aracteristics of the-desinn . The desinn calls for stepwise increr-ents ^ 
or decrerents in sir^ulation wi^th an eoual nunbor of observations of nerformance 
at Rach steo. Tach ^tepjwill b^ spoken of aq a condif-ion of ^si nu la M on . 
Each condition is e i ther 'presont or absent, buf a steo-down or r,l-ep-uo 
arranner^nt- holds, whereby the presence of a particular condition implies 
that conditions up to fhat point are present,' or tonverselv, that conditions 
down to that ooint are absent. 

V/ith the- above arrannencnt of <^pts of Gondlticns under which pMotinq 
perfornance' is observed, an estinato-of the nuitinle linear r^nression co- 
efficients for predi'cinq a pilct^s nerforrance score on a qiven nancuver , 
under a niven set of conditions can he obtained vory easily fron the 
difference of his scores under adjacent sots of conditions, ^uch estimates 
can be extended to differences arx)nn correspond i no scores when nore than one 
performance score is obtained for a eleven maneuver und6r different sets of 
conditions of simulation. \* 

When two or more observations of ! per fornance are obtain^^ foi; a sinnle 
pilot under each set of Conditions, the estimated multiple Imear reqression 
coefficients are qTven b]/ the mean of the difference scores under adjacent 
sets of conditions. Jfyfs also appHes to studies in which performances are 
observed under each sef of cond i t i ons • for two 'pr more oilots. The mean ^ 
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performance Is cQmputed for each set of conditions. Each estimated 
regression-coefficient is then equal to the difference In mean performance 
for a pair of adjacent sets of conditions. It might be noted here, even 
though the example given later will make this much clearer, that the step- 
up (Or step-down , If one wishes) arrangement of conditions Implies that If 
there are p conditions there will be p+l sets of conditions. Further, since 
the regression coefficients are based on the differences between'^two scores 
lhe_ same results ara obtained. -for raw^ scores -^s*- for scoi^^es-expressed in 
mean-deviation form.- Spechf1xa1|-v, (Y] - "Y2 E'CTj - T ) ^ (Y2 - 7 )]. 

An example of ^how' the design works . In the example which fol lows, 
there is a step^^up arranaement for four conditions of simulation. The five 
sets of conditions under which performance of a given piloting task Is 
observed on a single pilot can be represented by\ matrix, as follows: 
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Set of 
Conditions 



IF- 



I ro 



3 

"T" 



Conditions 
2 3 4 



II 



0 0 0 

0 0 0 

1 0 0 



'21 



'31 



22 



'32 



1 — 0- 



Yj|-is the performance 
score under the 1 
set of conditions 
for Trial I by a 
single pi lot, 
(i =.1,2.3,4,5). 
Y,^ Is simi iarly de- 
'"^ find for trl^l 7. ! 



'41 



42 



X 



I ' 0 I I ^31 II ^^52 I 



Is the 5x4 matrix 
whl~ch denotes the 
conditions under 
which performances 
are observed. 



















1 (Y21 




f|2 ^ 


' (Y22 


- Y|2) ^ 




b2i 


(Y3, 


- Y2I') _ 


•^22 


(Y32 


- ^22^ 




^31 


(■^41 




b32 


(Y42 


- ^32^ 




,^41 , 






•^42 1 


1 (Y52 


-.Y42) j 





regression coefficients, based on Trial L 
b2 is similarly defined for Trial ? 

b is the .4x1 vector of estimated regression coefficients, 
based on Trials I and 2. ^ 



6, 



= 1/2 (b_| + .b2) - 1/2 



(Y31 + Y32) . tY2| + Y22) (Y3. - Y2.) 



(Y41 + Y42) - (Y31 + Y32), 



(Y5, + Y52) - (Y4, + Y42) 



(■Y4. - Y3.f 



where Y. - is the mean of the performance scores under the 
th 

I set of conditions. 
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The least-squares prediction equation takes- the form:\ 
^ = X , where 

Y = , Y^ Is the lOxl vector of predicted performar^e , 



I scored Jn mean-deviation form; 

J 



X Is the 10x4 malr lx of precJ lrror-i73rTB& les in meah-deviat Ion form, 



representing the s^ts of 'coji.dltions undfir whjjcb- -the~obsen^ed--pei^f ormance - 

scores were obtained; and, b is the 4x1 vector of regression coefficients. 
The ratio oi the variance o7 the predicted performance scores to the 
variance of the observed performance scores Is eaual to the squared co- 
efficient of multiple correlation. 

2 0^2 7 • - ' ^ 

Thus, . = Y where - = I " rr^S^I . 

' YJ234 % Y Y ; l^ri. ' ^ • 

"y 

number of observations (10 in the present ca,se). 

Due to the fact that the matrix of Independent or predictor variables 
forms whatMs called an equally-spaced simplex, the prediction equation Is 
simply , ^ /. 

= ' - , 

where 

Y. . Is the predicted performance score under the 1th set of 

'J ^ ! * 1 — ;th .... 



conditions pn the j"*"" trial, and 

^ th • . ' 

Yj^ Is the mean performance score under the 1 set of conditions. 

From the above it can perhaps be seen that If a pilot's performance under 
each set of conditions were Identical for both trials, Y|^ would be ^aual 
to Yjj and, as a consequence, Ry.I234 would be equal to I !00. ; \ 

\ 

The example, described above, could be applied just as readily ta two 
pilots for whom one performance score was obtained under each set of \ 
conditions, as In the present case of one pilot for whom two performance 
scores were obtained under each set of conditions. The extension to more than 
four conditions of simulation is straightforward, as long .as the same number 
of observations of performance Is obtained for each of the p+l sets of 
conditions. When this Is not the case, the multiple linear prediction 
equation is not as simple as In the example given. Specifically, a predicted 
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scorb under the 1th set of conditions would then be based on observed 
performances untJer the (1-1), 1, and (1 + 1) sets of conditions, except 
for 1 = 1, In which case only I and (l + l) apply and l=p+K In which case 
only (M-l) and 1 apply, ^ • , . 



Further apprtcatlons of"the slmi 



ptex dggl'qn# An Interesting, and 
f the simp lei design would be one 



perhaps highly useful , application of the simp lei design 
wfiTch tfie coridttToTis of simulation are permtttedi-a vary so that equal 
Increments. In performance are obtained. Tte. regression coefficients 
would^^then be equal 4n value and the question of Interest would be to 
ascertain the conditions of simulation e.g., the k values which range from 
0 to l^for components of motion simulation, required for attaining equal 
Increments (or decrements) In observed performances. The usual model for 
multiple linear regression would apply directly, except that the Increments 
In performance would be required to be equal and the conditions of 
simulation would become the dependent variables. 

Finally, It Is worthy of note"*that the simplex design as described 
In this paper Is a fully-crossed design Inasmuch as each pilot's 
performance Is observed under all sets of conditions, trials, and 'jfasks. 
The resulting data for a given performance score (dependent variable) 
lend themselves to an analysis of variance whlQh leads to estimated 
components of variance. Computer programs are already avall^le"^ for ^ 
such analyses or can be readily developed. 



2. Cornelius, E. T., Woodward, J. A., and Demaree, R.*G., CRONB: 
A Fortran IV program to compute variance components for various 
— -ex p^Pf^nt^ designs. Fort Worth, Tex.: m^tltute of Behavioral 
Research, Texas Cnristian University, March; 1972. 
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LIST OF TRAINING PROBLEM PANEL MEMBERS 



of. Paul W. Caro 

Human Resources Research Office (Division 6) 
P. 0. Box 428 

Ft. Rucker, Ala. 36360 ' 

""Prof. Kaipn E.^H lexman 
Director, Institute of Aviation 
University of 1 1 1 Ino.ls 
Wlllard Airport 
Savoy, 111 61874 

Dr. Robert C. Houston, Director 
•Training Equipment/Program Development 
American Airlines Flight Academy 
Greater Southwest Airport 
Fort Worth, Tex. 71625 ' 

Mr. Dave KM I Ian 

Training Equipment/Program Development 
American Airlines Flight Academy 
Greater Southwest Airport 
Fort Worth, Tex. 71625 

Dr. George E. Long 

McDonnell Douglas Astronautics Cpmpany - East 
St. Louis, Missouri 63166 . 

Dr. Don E. Meyer 

Headquarters Air Training Command 
Randolph Air Force Base,. Tex. 78148 

Mr. Robert J. Ontlveros \ ^ - 

Department of Transportation ' 
Federal Aviation Administration 
National Aviation Facilities Experimental Center 
Atlantic City, New Jersey 08405 

Dr. Malcolm L. Ritchie 
Ritchie, Inc. 
630 Brubaker Drive 
Dayton, Ohio 45429 

Mr. Benjamin Schohan , 
Missile Systems Division 
Rockwel I International 
4300 E. Fifth Avenue 
Columbus, Ohio 43081 
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Dr. Edward A. Stark 
Singer Company 
Simulation Products Division 
BInghamton, New York 13905 

Mr, Donald - Vreui s, President 
Canyon Research Group, InCt " 

Suite 123 ■ 

32107 trndero Canyon Rd, 
Westlake Village, Calif. 91361 
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